Repolode
Submit
OpenBMB

OpenBMB/MiniCPM-V

Open-source multimodal models (MiniCPM-V 4.7 and MiniCPM-o 4.5) for understanding images, video, documents and speech on local and on-device hardware.

Open-source alternative to:GPT-4oGemini

View on GitHub
Stars
26,892
Forks
2,111
Open issues
64
Last commit
4 days ago

What is it?

MiniCPM-V and MiniCPM-o are open-source multimodal large language model families from THUNLP and ModelBest. MiniCPM-V 4.7 handles image, multi-image, document and video understanding, and comes in a 35B-A3B mixture-of-experts variant and a 1B variant. MiniCPM-o 4.5 is a 9B end-to-end omnimodal model that receives video and audio streams while generating speech and text, supporting real-time full-duplex conversation.

The project targets running these models on local hardware and phones. It describes on-device MoE inference using SSD-backed expert weights with Lookahead Expert Scheduling, and provides quantized weights in BF16, FP8, int4, AWQ, GPTQ and GGUF formats. Deployment is covered for iOS, Android and HarmonyOS, and for several chip platforms including Intel, AMD, Arm, Rockchip and Houmo.

Model weights are published on Hugging Face and ModelScope, with an Ollama listing and an API. The repository also includes fine-tuning guidance, with Self-Replay Fine-Tuning for extending capabilities while limiting forgetting.

Key features

  • 01Image, multi-image, document and video understanding with MiniCPM-V 4.7.
  • 02Full-duplex real-time speech and video interaction with MiniCPM-o 4.5.
  • 0335B-A3B MoE and 1B variants, with on-device inference optimizations.
  • 04Quantized weights in int4, FP8, AWQ, GPTQ and GGUF formats.
  • 05Deployment on iOS, Android, HarmonyOS and several Intel, AMD, Arm and Rockchip chip platforms.
  • 06Hugging Face, ModelScope, Ollama and API access.

Use cases

  • Running a vision-language model on a phone or laptop without sending data to the cloud.
  • Building document or image question-answering features into a local application.
  • Creating real-time voice and video assistants that can speak while watching a stream.
  • Fine-tuning a multimodal model to add new capabilities with less forgetting of existing ones.

Quick start

  1. 1

    Get the model weights from Hugging Face, ModelScope, or Ollama (links in the Model Weights table)

  2. 2

    Read the get-started section of the README for run instructions

Things to know

  • The 35B-A3B variant is large and needs substantial memory, and on-device use relies on SSD expert offloading.
  • The README excerpt shows no direct package install command, so setup depends on the linked model pages and docs.

Topics

#multimodal#vision-language-model#on-device-ai#speech#llm#moe#fine-tuning#ollama#minicpm#minicpm-o

FAQ

What is MiniCPM-V?
Open-source multimodal models (MiniCPM-V 4.7 and MiniCPM-o 4.5) for understanding images, video, documents and speech on local and on-device hardware. MiniCPM-V and MiniCPM-o are open-source multimodal large language model families from THUNLP and ModelBest. MiniCPM-V 4.7 handles image, multi-image, document and video understanding, and comes in a 35B-A3B mixture-of-experts variant and a 1B variant. MiniCPM-o 4.5 is a 9B end-to-end omnimodal model that receives video and audio streams while generating speech and text, supporting real-time full-duplex conversation.
Is MiniCPM-V free to use? What license does it have?
MiniCPM-V is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is MiniCPM-V still maintained?
Actively maintained. The last commit to MiniCPM-V was on Oct 16, 2026.
What can I use MiniCPM-V for?
Running a vision-language model on a phone or laptop without sending data to the cloud. Building document or image question-answering features into a local application. Creating real-time voice and video assistants that can speak while watching a stream. Fine-tuning a multimodal model to add new capabilities with less forgetting of existing ones.
How do I install MiniCPM-V?
Get the model weights from Hugging Face, ModelScope, or Ollama (links in the Model Weights table) Read the get-started section of the README for run instructions
Where can I get MiniCPM-V?
The source code and installation instructions are on GitHub: https://github.com/OpenBMB/MiniCPM-V

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects