Repolode
Submit
SomeOddCodeGuy

SomeOddCodeGuy/WilmerAI

A self-hosted LLM semantic router that sits between your app and multiple LLM APIs, running multi-step prompt workflows behind a standard OpenAI or Ollama-compatible endpoint.

View on GitHub
Stars
830
Forks
52
Open issues
3
Last commit
14 days ago

What is it?

WilmerAI routes requests to different LLMs based on the full conversation history rather than only the latest message. Routing and processing are defined as node-based JSON workflows, where each node can call a different LLM endpoint, run a tool, execute a script, or invoke another workflow. The client application sees the whole multi-step process as a single standard API call.

The project exposes OpenAI- and Ollama-compatible chat and generate endpoints, so existing front ends such as Open WebUI and SillyTavern can connect without modification. On the backend it connects to Claude, OpenAI-compatible servers, Ollama, and KoboldCpp. It also includes a four-part conversation memory system and experimental MCP tool calling.

WilmerAI does not track token usage, so operators should monitor costs through their LLM providers' dashboards. Output quality depends heavily on the connected models and prompts.

Key features

  • 01Routes prompts by full conversation context using configurable workflows.
  • 02Chains multiple LLMs and tools within one workflow via node-based JSON definitions.
  • 03Exposes OpenAI- and Ollama-compatible endpoints to existing front ends.
  • 04Connects to Claude, OpenAI-compatible, Ollama, and KoboldCpp backends.
  • 05Provides conversation memory with keyword and optional embedding-based search.
  • 06Supports OpenAI-style tool calling passthrough and JSON Schema structured output.

Use cases

  • Routing different types of chat requests to specialized local or cloud models.
  • Adding retrieval or web lookups to a chatbot before the final LLM call.
  • Using older machines as worker LLMs in a multi-model pipeline.
  • Keeping prompt logic on your own server while front ends stay unchanged.

Things to know

  • The project is still under development and is provided as-is without warranty.
  • It is licensed under GPL-3.0, a copyleft license that requires derivative works to remain open source.
  • The web fetcher requires Python 3.13.14/3.14.5 or newer; other features run on Python 3.11.10/3.12.13.
  • Agentic MCP tool integration is experimental.

In collections

Self-hosted appsRun LLMs locally

Topics

#llm#semantic-routing#workflow#prompt-engineering#openai-compatible#ollama#self-hosted#python#ai#generative-ai

FAQ

What is WilmerAI?
A self-hosted LLM semantic router that sits between your app and multiple LLM APIs, running multi-step prompt workflows behind a standard OpenAI or Ollama-compatible endpoint. WilmerAI routes requests to different LLMs based on the full conversation history rather than only the latest message. Routing and processing are defined as node-based JSON workflows, where each node can call a different LLM endpoint, run a tool, execute a script, or invoke another workflow. The client application sees the whole multi-step process as a single standard API call.
Is WilmerAI free to use? What license does it have?
WilmerAI is open source under the GPL-3.0 license. Check the license terms before using it in commercial products.
Is WilmerAI still maintained?
Actively maintained. The last commit to WilmerAI was on Oct 25, 2026.
What can I use WilmerAI for?
Routing different types of chat requests to specialized local or cloud models. Adding retrieval or web lookups to a chatbot before the final LLM call. Using older machines as worker LLMs in a multi-model pipeline. Keeping prompt logic on your own server while front ends stay unchanged.
Where can I get WilmerAI?
The source code and installation instructions are on GitHub: https://github.com/SomeOddCodeGuy/WilmerAI

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects