Repolode
Submit
jingyaogong

jingyaogong/minimind-v

A tiny open-source vision-language model (about 65M parameters) that you can train from scratch on a single GPU in roughly two hours.

View on GitHubWebsite
Stars
8,775
Forks
983
Open issues
22
Last commit
30 days ago

What is it?

MiniMind-V is a small multimodal model that pairs a vision encoder with a compact language model so it can answer questions about images. The project covers the full pipeline in plain PyTorch: model structure, dataset preparation, pretraining, and supervised fine-tuning, with inference and a WebUI included.

The vision side uses a SigLIP2 encoder that is kept frozen, while a small MLP projector maps its 64 patch features into the language model's token space. The language backbone is initialized from the companion MiniMind text model, and fine-tuning updates only the projector plus the first and last LLM layers. Released variants include a dense 65M model and a mixture-of-experts version.

The repository is aimed at learning how vision-language models are built, and it supports multi-GPU DDP training, bfloat16 mixed precision, checkpoint resumption, and wandb or SwanLab logging. Outputs are coarse, with noticeable repetition and detail errors, so it suits experimentation rather than production use.

Key features

  • 01Trains a 65M-parameter VLM from scratch with a documented SFT workflow on a single GPU.
  • 02Uses a frozen SigLIP2 vision encoder and an MLP projector mapping 64 image tokens into the LLM.
  • 03Provides dense and mixture-of-experts model variants.
  • 04Includes data preparation scripts for Parquet image-text datasets.
  • 05Supports DDP multi-GPU training, bfloat16, checkpoint resume, and wandb or SwanLab logging.
  • 06Ships command-line evaluation and a WebUI for chatting with images.

Use cases

  • Learning how a vision-language model is assembled from a vision encoder, projector, and LLM.
  • Running a small multimodal training experiment on a single consumer GPU.
  • Prototyping image question answering with a minimal, readable codebase.
  • Using it as a base for adapting a tiny VLM to a custom image-text dataset.

Topics

#vision-language-model#multimodal#pytorch#llm-training#small-models#siglip#image-captioning#artificial-intelligence#chatgpt

FAQ

What is minimind-v?
A tiny open-source vision-language model (about 65M parameters) that you can train from scratch on a single GPU in roughly two hours. MiniMind-V is a small multimodal model that pairs a vision encoder with a compact language model so it can answer questions about images. The project covers the full pipeline in plain PyTorch: model structure, dataset preparation, pretraining, and supervised fine-tuning, with inference and a WebUI included.
Is minimind-v free to use? What license does it have?
minimind-v is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is minimind-v still maintained?
Actively maintained. The last commit to minimind-v was on Sep 22, 2026.
What can I use minimind-v for?
Learning how a vision-language model is assembled from a vision encoder, projector, and LLM. Running a small multimodal training experiment on a single consumer GPU. Prototyping image question answering with a minimal, readable codebase. Using it as a base for adapting a tiny VLM to a custom image-text dataset.
Where can I get minimind-v?
The source code and installation instructions are on GitHub: https://github.com/jingyaogong/minimind-v

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects