Repolode
Submit
ZJUI-AI4H

ZJUI-AI4H/Hulu-Med

An open medical vision-language model that handles medical text, 2D images, 3D CT/MRI volumes and surgical videos, trained on public data.

View on GitHub
Stars
878
Forks
34
Open issues
23
Last commit
2 months ago

What is it?

Hulu-Med is a medical vision-language model from ZJU-AI4H designed to understand medical text, 2D images, 3D volumes such as NIfTI CT and MRI scans, and video, including surgical footage, within one model. It is released in several sizes, from 4B to 235B parameters, built on Qwen-family language backbones with a SigLIP-based vision encoder.

The project emphasizes openness: the authors describe the data curation, training code and model weights as a complete pipeline, and state that training uses public data. Models are available on Hugging Face and ModelScope, and can be loaded with the Transformers library using trust_remote_code, or run through a custom loader and a vLLM build for faster inference with tensor parallelism.

The repository includes example code for single-image, multi-image, interleaved, 3D and video inputs, plus an optional thinking mode that exposes the model's reasoning. It also links an evaluation toolkit, MedUniEval, for benchmarking medical vision-language models.

Key features

  • 01Handles medical text, 2D images, 3D NIfTI volumes and video in one model
  • 02Released in multiple sizes from 4B to 235B parameters
  • 03Loadable through Hugging Face Transformers with trust_remote_code
  • 04vLLM support with tensor parallelism for faster inference
  • 05Optional thinking mode that returns the model's reasoning

Use cases

  • Generating draft medical reports from 2D or 3D scans
  • Answering visual questions about pathology slides or radiology images
  • Analyzing surgical videos for phase recognition
  • Research benchmarking of medical multimodal models

Topics

#medical-ai#vision-language-model#multimodal#healthcare#transformers#vllm#medical-imaging

FAQ

What is Hulu-Med?
An open medical vision-language model that handles medical text, 2D images, 3D CT/MRI volumes and surgical videos, trained on public data. Hulu-Med is a medical vision-language model from ZJU-AI4H designed to understand medical text, 2D images, 3D volumes such as NIfTI CT and MRI scans, and video, including surgical footage, within one model. It is released in several sizes, from 4B to 235B parameters, built on Qwen-family language backbones with a SigLIP-based vision encoder.
Is Hulu-Med free to use? What license does it have?
Hulu-Med is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is Hulu-Med still maintained?
Updated within 6 months. The last commit to Hulu-Med was on Aug 30, 2026.
What can I use Hulu-Med for?
Generating draft medical reports from 2D or 3D scans Answering visual questions about pathology slides or radiology images Analyzing surgical videos for phase recognition Research benchmarking of medical multimodal models
Where can I get Hulu-Med?
The source code and installation instructions are on GitHub: https://github.com/ZJUI-AI4H/Hulu-Med

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects