Repolode
Submit
Osmantic

Osmantic/ODS

A self-hosted installer and stack that sets up a private local AI server on a PC, Mac, or Linux machine with chat, voice, RAG, workflows, and image generation.

Open-source alternative to:ChatGPT

View on GitHubWebsite
Stars
7,780
Forks
1,109
Open issues
929
Last commit
15 hours ago

What is it?

ODS (Osmantic Deployment System) installs and connects the pieces needed to run AI models on your own hardware, including local inference through llama-server, Open WebUI chat, a control dashboard, Whisper and Kokoro for speech, n8n workflows, Qdrant-based RAG, SearXNG search, and ComfyUI for image generation. A single install command detects the GPU or Apple Silicon chip, selects a model that fits the available memory, generates credentials, and starts the services in Docker.

The installer can start a small model while the full model downloads in the background. Each service is packaged as an extension with a manifest and Compose file, so services can be enabled or disabled through the ods CLI. The CLI also switches between local, cloud, and hybrid API modes via LiteLLM.

The project supports Linux, Windows through WSL2 and Docker Desktop, and macOS on Apple Silicon. It is labeled V3 pre-release: the installers track the main branch, and the project documents known limits and validation status separately.

Key features

  • 01One-command installer that detects GPU or Apple Silicon and picks a model for the available memory
  • 02Open WebUI chat interface with llama-server local inference
  • 03Control dashboard for models, services, GPU status and extensions
  • 04Voice support with Whisper speech-to-text and Kokoro text-to-speech
  • 05n8n workflow automation, Qdrant vector search and SearXNG web search
  • 06Extension system with an ods CLI to enable, disable and manage services

Use cases

  • Running a private chat and document assistant at home without cloud API subscriptions
  • Setting up a local AI workstation or homelab with RAG over your own files
  • Building voice and workflow automations that stay on your own machine
  • Generating images locally with ComfyUI without sending prompts to a hosted service

Quick start

  1. 1

    Install Docker and make sure it is running (Linux or macOS)

  2. 2

    Run the official install script from the README (Linux or macOS)

  3. 3

    On Windows, run the PowerShell installer block from the README in a normal PowerShell window

Things to know

  • This is a V3 pre-release installing the unsigned main branch, so it may change often.
  • Windows setup requires WSL2, Docker Desktop and about 40 GB of free disk space.
  • Linux with Intel Arc support is experimental and hardware-specific.
  • Cloud mode depends on external API services such as OpenAI, Anthropic or Together.

In collections

Open-source AI agent frameworks & appsSelf-hosted appsRun LLMs locallyAI video & audio tools

Topics

#local-ai#llm#self-hosted#docker#rag#text-to-speech#speech-to-text#workflow-automation#ai-agents#amd

FAQ

What is ODS?
A self-hosted installer and stack that sets up a private local AI server on a PC, Mac, or Linux machine with chat, voice, RAG, workflows, and image generation. ODS (Osmantic Deployment System) installs and connects the pieces needed to run AI models on your own hardware, including local inference through llama-server, Open WebUI chat, a control dashboard, Whisper and Kokoro for speech, n8n workflows, Qdrant-based RAG, SearXNG search, and ComfyUI for image generation. A single install command detects the GPU or Apple Silicon chip, selects a model that fits the available memory, generates credentials, and starts the services in Docker.
Is ODS free to use? What license does it have?
ODS is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is ODS still maintained?
Actively maintained. The last commit to ODS was on Nov 6, 2026.
What can I use ODS for?
Running a private chat and document assistant at home without cloud API subscriptions Setting up a local AI workstation or homelab with RAG over your own files Building voice and workflow automations that stay on your own machine Generating images locally with ComfyUI without sending prompts to a hosted service
How do I install ODS?
Install Docker and make sure it is running (Linux or macOS) Run the official install script from the README (Linux or macOS) On Windows, run the PowerShell installer block from the README in a normal PowerShell window
Where can I get ODS?
The source code and installation instructions are on GitHub: https://github.com/Osmantic/ODS

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects