Repolode
Submit
alibaba

alibaba/MNN

MNN is a lightweight deep learning inference and training engine from Alibaba, designed for running models on mobile and embedded devices, including on-device LLMs.

Open-source alternative to:TensorFlow Lite

View on GitHub
Stars
16,256
Forks
2,482
Open issues
75
Last commit
7 hours ago

What is it?

MNN is a deep learning framework from Alibaba that supports both inference and training, with a focus on running models efficiently on devices rather than in the cloud. It converts models from TensorFlow, Caffe, ONNX and TorchScript into its own format and runs them on CPU and GPU backends including ARM, x86, Metal, OpenCL, Vulkan and CUDA.

The project also includes MNN-LLM and MNN-Diffusion runtimes for deploying large language models and Stable Diffusion locally on phones, PCs and IoT hardware. Supporting tools cover model conversion, compression, a Python API, and an OpenCV-like image processing library.

The README describes the engine as being integrated into more than 30 Alibaba apps and used on embedded devices such as IoT hardware. It targets iOS 8.0+, Android 4.3+ and POSIX-based embedded systems.

Key features

  • 01Converts TensorFlow, Caffe, ONNX and TorchScript models into MNN format.
  • 02Runs inference on CPU (ARM, x86) and GPU backends including Metal, OpenCL, Vulkan and CUDA.
  • 03Supports FP16 and Int8 quantization to reduce model size.
  • 04Provides MNN-LLM runtime for running large language models on local devices.
  • 05Offers Python API and a lightweight OpenCV-like image processing module.

Use cases

  • Running LLMs or diffusion models offline on Android or iOS apps.
  • Deploying computer vision models on embedded or IoT hardware.
  • Accelerating ONNX or TensorFlow models on mobile GPUs without a cloud backend.

Quick start

  1. 1

    Read the MNN documentation on Read the Docs

  2. 2

    Download MNN Workbench from the MNN homepage (www.mnn.zone)

Things to know

  • Support varies by backend: several CPU, GPU and NPU combinations are marked as buggy, unoptimized, or unsupported in the README.
  • The community discussion is mainly in Chinese, with limited English support.

Topics

#deep-learning#inference-engine#on-device-ai#llm#edge-ai#mobile#arm#gpu#convolution#embedded-devices

FAQ

What is MNN?
MNN is a lightweight deep learning inference and training engine from Alibaba, designed for running models on mobile and embedded devices, including on-device LLMs. MNN is a deep learning framework from Alibaba that supports both inference and training, with a focus on running models efficiently on devices rather than in the cloud. It converts models from TensorFlow, Caffe, ONNX and TorchScript into its own format and runs them on CPU and GPU backends including ARM, x86, Metal, OpenCL, Vulkan and CUDA.
Is MNN free to use? What license does it have?
MNN is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is MNN still maintained?
Actively maintained. The last commit to MNN was on Oct 21, 2026.
What can I use MNN for?
Running LLMs or diffusion models offline on Android or iOS apps. Deploying computer vision models on embedded or IoT hardware. Accelerating ONNX or TensorFlow models on mobile GPUs without a cloud backend.
How do I install MNN?
Read the MNN documentation on Read the Docs Download MNN Workbench from the MNN homepage (www.mnn.zone)
Where can I get MNN?
The source code and installation instructions are on GitHub: https://github.com/alibaba/MNN

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects