Open-source projects tagged “multimodal”

Open-source native vision-language models (NEO series) that process pixels and words in one encoder-free dense architecture, with 2B and 9B releases.

Open-source Python package that turns websites, PDFs, images, YouTube videos or topics into multilingual audio podcast conversations using GenAI.

A unified training engine for multimodal models, supporting vision-language, diffusion, and MoE architectures with FSDP2, Muon, and Liger kernels.

A video LLM trained with streaming speech transcription so it can produce real-time commentary on video, with Gradio and CLI demos.

Self-hostable captcha solver exposing a YesCaptcha-style createTask/getTaskResult API, built on FastAPI, Playwright and OpenAI-compatible vision models.

Turns spare Android phones into LAN camera nodes, with a PC dashboard for live MJPEG preview, recording, and optional AI-triggered event analysis.

Converts PDFs, Office files, scans, screenshots and diagrams into Markdown by sending pages to your own OpenAI-compatible vision model.

Open-source native multimodal models that handle image understanding, text-to-image generation, and image editing within one unified architecture.

Visual RAG toolkit that renders web pages and PDFs as screenshots and retrieves over the images, keeping tables and charts intact.

A small educational project for running vision-language model inference on Apple Silicon with MLX, scoring candidate answers about one image.

A 9B-parameter open-weight multimodal model for visual understanding, spatial reasoning, tool use, and embodied-AI research, built on Qwen3.5 and C-RADIOv4-H.

A unified multimodal model that handles many computer vision tasks by generating text, images, or both from natural-language instructions.

Desktop app that searches your local files by meaning, across photos, PDFs, documents, audio, video and code, running on-device with EmbeddingGemma 2.