Repolode
Submit
mudler

mudler/LocalAI

A self-hosted, OpenAI-compatible AI engine that runs LLMs, vision, voice, image and video models on any hardware, including CPU-only machines.

Open-source alternative to:OpenAI API

View on GitHubWebsite
Stars
49,633
Forks
4,507
Open issues
202
Last commit
14 minutes ago

What is it?

LocalAI is an open-source engine for serving AI models on your own infrastructure. It exposes OpenAI, Anthropic and ElevenLabs compatible APIs, so existing client code can point at a local server instead of a hosted service. It covers text generation, vision, speech-to-text and text-to-speech, image and video generation, embeddings, reranking and object detection behind one API.

The core stays small. Each backend wraps an engine such as llama.cpp, vLLM, whisper.cpp, stable-diffusion or MLX and is installed as its own image only when a model needs it. Hardware support includes NVIDIA CUDA, AMD ROCm, Intel oneAPI, Apple Silicon Metal, Vulkan and CPU-only operation.

Beyond inference, LocalAI includes API key authentication, per-user quotas, built-in agents with tool use and RAG, MCP support, and a distributed mode that uses PostgreSQL and NATS for horizontal scaling.

Key features

  • 01OpenAI, Anthropic and ElevenLabs compatible API endpoints across backends
  • 02Runs LLMs, vision, speech, image and video models behind one API
  • 03Backends install on demand, so only the engines a model needs are pulled
  • 04Supports NVIDIA, AMD, Intel, Apple Silicon, Vulkan and CPU-only hardware
  • 05Built-in agents with tool use, RAG and MCP support
  • 06API key auth, per-user quotas and role-based access for multi-user setups

Use cases

  • Running a private, self-hosted replacement for hosted OpenAI-style APIs
  • Serving open models on a workstation or server without a dedicated GPU
  • Building voice assistants that combine speech-to-text, an LLM and text-to-speech locally
  • Operating a shared model service for a team with per-user quotas and API keys

Topics

#llm#openai-compatible#self-hosted#inference#multimodal#rag#text-to-speech#image-generation#agents#ai

FAQ

What is LocalAI?
A self-hosted, OpenAI-compatible AI engine that runs LLMs, vision, voice, image and video models on any hardware, including CPU-only machines. LocalAI is an open-source engine for serving AI models on your own infrastructure. It exposes OpenAI, Anthropic and ElevenLabs compatible APIs, so existing client code can point at a local server instead of a hosted service. It covers text generation, vision, speech-to-text and text-to-speech, image and video generation, embeddings, reranking and object detection behind one API.
Is LocalAI free to use? What license does it have?
LocalAI is open source under the MIT license. Check the license terms before using it in commercial products.
Is LocalAI still maintained?
Actively maintained. The last commit to LocalAI was on Oct 19, 2026.
What can I use LocalAI for?
Running a private, self-hosted replacement for hosted OpenAI-style APIs Serving open models on a workstation or server without a dedicated GPU Building voice assistants that combine speech-to-text, an LLM and text-to-speech locally Operating a shared model service for a team with per-user quotas and API keys
Where can I get LocalAI?
The source code and installation instructions are on GitHub: https://github.com/mudler/LocalAI

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects