Open-source projects tagged “on-device-ai”

Swift and Metal runtime that runs Gemma 4 26B-A4B on Apple Silicon Macs with about 2 GB of RAM by streaming experts from SSD.

A macOS and iOS document chat app that answers questions about your files using Apple Foundation Models, running entirely on-device.

Runs Claude Code against open AI models on your Apple Silicon Mac via an MLX-based Anthropic API server, with no cloud model or API key.

A 8-29 MB foundation model for phones, wearables, robots and microcontrollers that handles tool calls, structured extraction, embeddings and speech-to-text on-device.

C++ edge AI engine that turns video analytics and on-device VLM models into deployable apps on Sophon, Rockchip RKNN, and x86 hardware.