Repolode
Submit
av

av/harbor

A Docker Compose-based CLI and companion app that spins up a pre-wired local LLM stack, from backends like Ollama and llama.cpp to frontends like Open WebUI, with one command.

View on GitHubWebsite
Stars
3,263
Forks
234
Open issues
52
Last commit
7 days ago

What is it?

Harbor is a CLI and companion desktop app for running a complete local LLM stack. You choose the services you want, such as inference backends like Ollama, llama.cpp, or vLLM, frontends like Open WebUI, or supporting services like SearXNG for web search and Speaches for voice chat, and Harbor handles the Docker Compose orchestration, configuration, and connectivity between them.

The project covers a wide range of services grouped into UIs, backends, and satellite tools, including ComfyUI for image generation, n8n and Dify for workflows, and MCP-related tools such as MetaMCP and mcpo. On macOS, some backends such as Docker Model Runner, MLX, and oMLX run inference on the host with Metal acceleration rather than inside containers.

Harbor also includes a `harbor launch` command that connects coding and agent CLIs such as Codex, Claude, and OpenCode to a Harbor backend, along with Boost workflows, config profiles, a built-in tunnel for remote access, and an eject command that exports the setup as a standalone Docker Compose file.

Key features

  • 01Starts pre-connected inference backends and frontends with a single `harbor up` command.
  • 02Connects supporting services such as SearXNG web search and Speaches voice chat to Open WebUI automatically.
  • 03Runs host-native inference on macOS with Metal acceleration for Docker Model Runner, MLX, and oMLX.
  • 04Launches coding and agent CLIs against a Harbor backend without manual provider config.
  • 05Saves and switches configuration profiles and command history.
  • 06Exports the current setup into a standalone Docker Compose file.

Use cases

  • Setting up a local Open WebUI and llama.cpp or Ollama environment on a personal machine in one step.
  • Adding web-search-backed RAG or voice chat to a local model without wiring services by hand.
  • Comparing several inference backends side by side with the same frontend.
  • Moving a tested local setup into a standalone Docker Compose deployment.

Quick start

  1. 1

    Install the Harbor CLI and App using the installation guide

  2. 2

    Start a fully configured Open WebUI and llama.cpp stack

    harbor up
  3. 3

    Add SearXNG web search and Speaches voice chat to Open WebUI

    harbor up searxng speaches

Things to know

  • Requires Docker, since the stack runs on Docker Compose.
  • Running local models well needs suitable hardware, such as a GPU, and the full stack of many services can be heavy.

In collections

MCP serversSelf-hosted appsRun LLMs locally

Topics

#llm#docker-compose#self-hosted#local-llm#ollama#llama-cpp#open-webui#mcp#ai#automation

FAQ

What is harbor?
A Docker Compose-based CLI and companion app that spins up a pre-wired local LLM stack, from backends like Ollama and llama.cpp to frontends like Open WebUI, with one command. Harbor is a CLI and companion desktop app for running a complete local LLM stack. You choose the services you want, such as inference backends like Ollama, llama.cpp, or vLLM, frontends like Open WebUI, or supporting services like SearXNG for web search and Speaches for voice chat, and Harbor handles the Docker Compose orchestration, configuration, and connectivity between them.
Is harbor free to use? What license does it have?
harbor is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is harbor still maintained?
Actively maintained. The last commit to harbor was on Oct 31, 2026.
What can I use harbor for?
Setting up a local Open WebUI and llama.cpp or Ollama environment on a personal machine in one step. Adding web-search-backed RAG or voice chat to a local model without wiring services by hand. Comparing several inference backends side by side with the same frontend. Moving a tested local setup into a standalone Docker Compose deployment.
How do I install harbor?
Install the Harbor CLI and App using the installation guide Start a fully configured Open WebUI and llama.cpp stack: harbor up Add SearXNG web search and Speaches voice chat to Open WebUI: harbor up searxng speaches
Where can I get harbor?
The source code and installation instructions are on GitHub: https://github.com/av/harbor

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects