Repolode
Submit
rafska

rafska/awesome-local-llm

A curated awesome-list of platforms, engines, UIs, models, tools and tutorials for running large language models locally.

View on GitHub
Stars
2,984
Forks
498
Open issues
234
Last commit
3 days ago

What is it?

awesome-local-llm is a curated list of resources for running large language models on your own hardware instead of through a hosted API. It is organized by category: inference platforms and engines such as Ollama, llama.cpp and vLLM, user interfaces like Open WebUI and SillyTavern, and collections of open-weight models from providers including Qwen, Mistral, Gemma and NVIDIA.

Beyond software, the list covers model leaderboards and benchmarks, agent frameworks, Model Context Protocol tools, retrieval-augmented generation components, hardware guidance, and tutorials on prompt and context engineering and inference. Entries link directly to project pages or Hugging Face collections, with short descriptions to help readers pick a tool quickly.

Key features

  • 01Covers inference engines such as Ollama, llama.cpp, vLLM and MLX-based tools
  • 02Lists desktop and web user interfaces for chatting with local models
  • 03Gathers open-weight model families grouped by use case, such as coding, vision and speech
  • 04Includes agent frameworks, RAG components and MCP tools for local setups
  • 05Provides benchmarks, leaderboards and hardware and tutorial resources

Use cases

  • Choosing an inference engine or chat interface before setting up a local LLM
  • Finding open-weight models suited to coding, vision or speech tasks
  • Discovering agent frameworks and RAG tools that work with self-hosted models
  • Learning about local LLM practices through linked tutorials

Topics

#llm#local-llm#awesome-list#self-hosted#inference#open-weight-models#ai-tools#ai#awesome#local

FAQ

What is awesome-local-llm?
A curated awesome-list of platforms, engines, UIs, models, tools and tutorials for running large language models locally. awesome-local-llm is a curated list of resources for running large language models on your own hardware instead of through a hosted API. It is organized by category: inference platforms and engines such as Ollama, llama.cpp and vLLM, user interfaces like Open WebUI and SillyTavern, and collections of open-weight models from providers including Qwen, Mistral, Gemma and NVIDIA.
Is awesome-local-llm free to use? What license does it have?
awesome-local-llm is open source under the MIT license. Check the license terms before using it in commercial products.
Is awesome-local-llm still maintained?
Actively maintained. The last commit to awesome-local-llm was on Oct 17, 2026.
What can I use awesome-local-llm for?
Choosing an inference engine or chat interface before setting up a local LLM Finding open-weight models suited to coding, vision or speech tasks Discovering agent frameworks and RAG tools that work with self-hosted models Learning about local LLM practices through linked tutorials
Where can I get awesome-local-llm?
The source code and installation instructions are on GitHub: https://github.com/rafska/awesome-local-llm

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects