Open-source projects tagged “local-llm”

Self-hosted AI research assistant that runs deep, multi-step research with local or cloud LLMs and searches the web, academic databases and your own documents.

A curated awesome-list of platforms, inference engines, UIs, models, tools and tutorials for running LLMs on your own hardware.

An edge-native MoE serving engine that runs large open-weight models like DeepSeek-V4 and Qwen3.6 on consumer GPUs by splitting work between GPU and CPU memory.

A local reasoning tool where a small model's step-by-step answers are stored as a knowledge graph, all running on your own machine via Ollama.

A curated Awesome-style list of platforms, tools, models and tutorials for running large language models locally.

Runs Claude Code against open AI models on your Apple Silicon Mac via an MLX-based Anthropic API server, with no cloud model or API key.

A hands-on tutorial series for building AI agents from scratch with local LLMs via node-llama-cpp, without using frameworks.

An educational JavaScript tutorial that builds retrieval-augmented generation step by step with local LLMs and no cloud APIs.

A pure C inference engine that runs very large mixture-of-experts models on ordinary hardware by streaming routed experts from disk.

CLI tool to fine-tune and post-train LLMs from a single YAML config, with layer streaming for training 8B models on a 4 GB GPU.

A local-first browser automation engine and Agent Skill that runs Qwen3.5 models on-device to pick browser actions from visible page elements.

Uses an iPhone connected over USB-C to speed up prefill and extend context for a 27B Qwen model running locally on a Mac.

Spark-X2.5 is an open-weight language model series (4B and 1.7B) aimed at agentic coding and tool use on local and on-device hardware.

Open source inference engine that tunes its kernels to your hardware to run each open model as fast as possible.

Cross-platform desktop app that uses local or remote LLMs to sort files into categories and suggest clearer filenames, with review before any change.

A Zotero plugin that lets you chat with one PDF, several PDFs, or your whole library, with clickable citations and optional fully local models.

Chrome extension plus local Python service that generates real-time English/Chinese translated captions for YouTube livestreams using Ollama.