Repolode
Submit
Tencent

Tencent/ncnn

A high-performance neural network inference framework in C++ for running deep learning models on mobile, embedded, and desktop devices.

Open-source alternative to:TensorFlow LiteCore ML

View on GitHub
Stars
23,985
Forks
4,532
Open issues
1,268
Last commit
yesterday

What is it?

ncnn is a neural network inference framework written in C++ for deploying trained models on phones, PCs, browsers, and edge devices. It has no third-party runtime dependencies and runs on CPU with ARM NEON optimizations, or on Vulkan GPUs.

Models from PyTorch and ONNX are converted with the pnnx tool into ncnn's .param and .bin format, then loaded through a C++, C, or Python API. The project supports multi-input and multi-output graphs, fp16 and int8 quantized inference, and custom layers. Build instructions cover Linux, Windows, macOS, Android, iOS, HarmonyOS, WebAssembly, and several embedded boards.

The project is used in several Tencent applications, including QQ, Qzone, and WeChat, according to its documentation. It also hosts examples for classic CNN models such as YOLO and MobileNet, along with newer workloads like LLM and image generation demos.

Key features

  • 01Runs on CPU with ARM NEON optimizations and multi-core scheduling, or on Vulkan GPUs.
  • 02Has no third-party runtime dependencies.
  • 03Converts PyTorch and ONNX models with the pnnx tool.
  • 04Provides C++, C, and Python APIs.
  • 05Supports fp16 and int8 quantized inference and custom layers.
  • 06Builds for Android, iOS, HarmonyOS, Windows, Linux, macOS, and WebAssembly.

Use cases

  • Running object detection or face detection models inside a mobile app.
  • Deploying a PyTorch model to an embedded board without a heavy runtime.
  • Running neural network inference in a browser via WebAssembly.
  • Adding AI features to a C++ desktop application.

Quick start

  1. 1

    Install pnnx in a PyTorch environment

    pip3 install pnnx
  2. 2

    Export a PyTorch model to ncnn with pnnx.export

  3. 3

    Load the generated model and run inference with the ncnn C++ or Python API

Things to know

  • Building ncnn for many platforms requires following the separate how-to-build wiki guide.
  • The metadata license is unassigned, although the README badge says BSD 3-Clause, so check LICENSE.txt before use.

Topics

#inference#deep-learning#mobile#neural-network#onnx#pytorch#vulkan#arm-neon#android#artificial-intelligence

FAQ

What is ncnn?
A high-performance neural network inference framework in C++ for running deep learning models on mobile, embedded, and desktop devices. ncnn is a neural network inference framework written in C++ for deploying trained models on phones, PCs, browsers, and edge devices. It has no third-party runtime dependencies and runs on CPU with ARM NEON optimizations, or on Vulkan GPUs.
Is ncnn free to use? What license does it have?
ncnn is open source under the Other license. Check the license terms before using it in commercial products.
Is ncnn still maintained?
Actively maintained. The last commit to ncnn was on Oct 20, 2026.
What can I use ncnn for?
Running object detection or face detection models inside a mobile app. Deploying a PyTorch model to an embedded board without a heavy runtime. Running neural network inference in a browser via WebAssembly. Adding AI features to a C++ desktop application.
How do I install ncnn?
Install pnnx in a PyTorch environment: pip3 install pnnx Export a PyTorch model to ncnn with pnnx.export Load the generated model and run inference with the ncnn C++ or Python API
Where can I get ncnn?
The source code and installation instructions are on GitHub: https://github.com/Tencent/ncnn

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects