Repolode
Submit
LostRuins

LostRuins/koboldcpp

Single-file local runner for GGUF language models with a built-in KoboldAI web UI, supporting text, image, speech and music generation.

Open-source alternative to:ChatGPTCharacter.AI

View on GitHubWebsite
Stars
12,189
Forks
807
Open issues
532
Last commit
6 hours ago

What is it?

KoboldCpp runs GGUF large language models on your own computer, on CPU or GPU, without installing anything. The download is a single executable for Windows, Linux and macOS. Once a model is loaded, it serves a bundled KoboldAI Lite web UI for chat, roleplay, adventure and story writing, along with a llama.cpp web UI and several other interfaces.

Beyond text generation, the project supports image generation and editing with models such as Stable Diffusion, SDXL and Flux, plus video generation, Whisper speech-to-text, and text-to-speech models such as Kokoro and Qwen3-TTS. It also includes MCP server support, tool calling, web search, and an integrated agent for writing code and scheduling tasks.

For integration, KoboldCpp exposes KoboldAI, OpenAI-compatible, Ollama, AUTOMATIC1111, ComfyUI and other API endpoints, so existing apps can connect to a local server. It can also forward requests to external providers such as OpenRouter, Gemini or Mistral when started without a local model.

Key features

  • 01Runs GGUF text models on CPU or GPU, with full or partial GPU offloading.
  • 02Ships as a single executable with no installation or external dependencies.
  • 03Includes KoboldAI Lite and llama.cpp web UIs with memory, world info and character cards.
  • 04Generates images, video, speech and music with compatible model files.
  • 05Offers OpenAI, Ollama, KoboldAI and other compatible API endpoints.

Use cases

  • Chatting with a private AI assistant that runs entirely on your own machine.
  • Roleplay and interactive fiction with characters, world info and saved stories.
  • Pointing existing OpenAI- or Ollama-compatible apps at a local model server.
  • Experimenting with local image, speech or music generation alongside text models.

Quick start

  1. 1

    Download the KoboldCpp executable for your OS from the latest release page

  2. 2

    Download a GGUF text model separately from Hugging Face

  3. 3

    Launch KoboldCpp with your model file (Linux example)

    ./koboldcpp-linux-x64 --model /path/to/model.gguf
  4. 4

    Open the Web UI in your browser at localhost port 5001

Things to know

  • Licensed under AGPL-3.0, a copyleft license that requires sharing modified source code when distributed or served over a network.
  • Model files are not included and must be downloaded separately.

In collections

Run LLMs locallyAI image generators & Stable Diffusion tools

Topics

#llm#gguf#local-llm#llama-cpp#text-generation#roleplay#image-generation#tts#gemma#ggml

FAQ

What is koboldcpp?
Single-file local runner for GGUF language models with a built-in KoboldAI web UI, supporting text, image, speech and music generation. KoboldCpp runs GGUF large language models on your own computer, on CPU or GPU, without installing anything. The download is a single executable for Windows, Linux and macOS. Once a model is loaded, it serves a bundled KoboldAI Lite web UI for chat, roleplay, adventure and story writing, along with a llama.cpp web UI and several other interfaces.
Is koboldcpp free to use? What license does it have?
koboldcpp is open source under the AGPL-3.0 license. Check the license terms before using it in commercial products.
Is koboldcpp still maintained?
Actively maintained. The last commit to koboldcpp was on Nov 7, 2026.
What can I use koboldcpp for?
Chatting with a private AI assistant that runs entirely on your own machine. Roleplay and interactive fiction with characters, world info and saved stories. Pointing existing OpenAI- or Ollama-compatible apps at a local model server. Experimenting with local image, speech or music generation alongside text models.
How do I install koboldcpp?
Download the KoboldCpp executable for your OS from the latest release page Download a GGUF text model separately from Hugging Face Launch KoboldCpp with your model file (Linux example): ./koboldcpp-linux-x64 --model /path/to/model.gguf Open the Web UI in your browser at localhost port 5001
Where can I get koboldcpp?
The source code and installation instructions are on GitHub: https://github.com/LostRuins/koboldcpp

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects