Open-source multimodal models (MiniCPM-V 4.7 and MiniCPM-o 4.5) for understanding images, video, documents and speech on local and on-device hardware.
26.9K2.1KPythonApache-2.0Updated 4 days ago
Taichu-AI/ZDTaichu5.0-9B
A 9B-parameter open-weight multimodal model for visual understanding, spatial reasoning, agentic tool use, and embodied-AI research, built on Qwen3.5 and C-RADIOv4-H.
Open-source alternative to:GPT-4V
- Stars
- 3,076
- Forks
- 540
- Open issues
- 3
- Last commit
- last month
What is it?
ZDTaichu5.0-9B is a multimodal foundation model that accepts text, one or more images, and video, with any-resolution visual input. It pairs a Qwen3.5-9B language backbone with a C-RADIOv4-H vision encoder. The model targets general image, document, chart, diagram and OCR understanding, as well as visual math, multi-view spatial reasoning, multi-step tool use, and spatial perception for VLA and embodied-AI work.
The project publishes several variants on Hugging Face and ModelScope, including FP8, NVFP4 and DSpark versions. Weights are available for download, and the model can be run with Hugging Face Transformers through trust_remote_code, or served through a vLLM fork that exposes an OpenAI-compatible chat completions endpoint with image and video input. Optional flags enable reasoning output and tool-call parsing.
The repository also describes an entropy-gated adaptive recurrent reasoning mechanism that allocates extra refinement steps in latent space to harder tokens. Model weights are released under the NVIDIA Open Model License, with Qwen3.5 Apache-2.0 and other upstream notices retained.
Key features
- 01Accepts text, multiple images and video as input, with any-resolution visual input.
- 02Covers document, chart, diagram and OCR understanding alongside visual math.
- 03Targets spatial reasoning tasks such as multi-view association and 3D scene understanding.
- 04Supports multi-step and multi-turn tool use.
- 05Provides FP8, NVFP4 and DSpark variants on Hugging Face and ModelScope.
- 06Offers an OpenAI-compatible serving path through a vLLM fork with Docker images.
Use cases
- Extracting information from documents, charts and diagrams with a vision-language model.
- Building agents that combine image understanding with multi-step tool calls.
- Researching spatial reasoning or embodied-AI perception and planning.
- Serving a local multimodal model behind an OpenAI-style chat API.
Things to know
- Benchmark results are the authors' own reported numbers under their stated evaluation settings, not independently verified here.
- The README excerpt gives model download links but no install or run commands, so setup steps are not documented in the visible text.
Topics
FAQ
- What is ZDTaichu5.0-9B?
- A 9B-parameter open-weight multimodal model for visual understanding, spatial reasoning, agentic tool use, and embodied-AI research, built on Qwen3.5 and C-RADIOv4-H. ZDTaichu5.0-9B is a multimodal foundation model that accepts text, one or more images, and video, with any-resolution visual input. It pairs a Qwen3.5-9B language backbone with a C-RADIOv4-H vision encoder. The model targets general image, document, chart, diagram and OCR understanding, as well as visual math, multi-view spatial reasoning, multi-step tool use, and spatial perception for VLA and embodied-AI work.
- Is ZDTaichu5.0-9B free to use? What license does it have?
- ZDTaichu5.0-9B is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
- Is ZDTaichu5.0-9B still maintained?
- Updated within 6 months. The last commit to ZDTaichu5.0-9B was on Sep 17, 2026.
- What can I use ZDTaichu5.0-9B for?
- Extracting information from documents, charts and diagrams with a vision-language model. Building agents that combine image understanding with multi-step tool calls. Researching spatial reasoning or embodied-AI perception and planning. Serving a local multimodal model behind an OpenAI-style chat API.
- Where can I get ZDTaichu5.0-9B?
- The source code and installation instructions are on GitHub: https://github.com/Taichu-AI/ZDTaichu5.0-9B
Summary generated by AI from the project's README. GitHub stats refresh daily.
Related projects
Open-weight gpt-oss-120b and gpt-oss-20b language models from OpenAI, with reference inference, tool, and client code.
20.5K2.2KPythonApache-2.0Updated 8 days ago
NEO is a family of native vision-language models that process pixels and words in one encoder-free dense architecture, with training and evaluation code.
90433PythonApache-2.0Updated 3 months ago
An open medical vision-language model that handles medical text, 2D images, 3D CT/MRI volumes and surgical videos, trained on public data.
87834PythonApache-2.0Updated 2 months ago
A unified training engine for multimodal models, supporting VLM, diffusion and LLM training with FSDP2, Muon and Liger kernels.
83238PythonNo licenseUpdated 2 days ago
A unified multimodal model that handles computer vision tasks like detection, OCR, segmentation, depth and multi-view geometry through text and image generation.
AI & Machine Learning#multimodal#computer-vision#vision-language-model#segmentation#depth-estimation
71742PythonApache-2.0Updated 2 months ago