Repolode
Submit
aurelio-labs

aurelio-labs/semantic-router

A decision layer for LLMs and agents that routes requests by semantic similarity or a small decision model, returning calibrated confidence.

View on GitHubWebsite
Stars
3,962
Forks
382
Open issues
138
Last commit
6 hours ago

What is it?

Semantic Router decides which route a user request belongs to without waiting for a full LLM generation. Each route is defined by example utterances; incoming queries are embedded and compared against them in vector space, so a decision takes milliseconds. Alternatively, a DecisionRouter asks a small decision model (Jev, Laya or Von) to pick the applicable route in a single call.

Confidence scores can be calibrated with fit() into probabilities, and a confidence threshold lets the router return None when no route applies. Routers can be combined with Chain, which falls through to a slower router when unsure, and Merge, which averages several routers. Hybrid search via a sparse encoder, Pinecone, Qdrant or Postgres indexes, and image-based multi-modal routes are also supported, with encoders from OpenAI, Cohere, Hugging Face and FastEmbed for fully local use.

Key features

  • 01Routes requests by semantic similarity using example utterances and vector embeddings.
  • 02Offers a DecisionRouter backed by small decision models for single-call route selection.
  • 03Calibrates confidence scores into probabilities with fit() and a confidence threshold.
  • 04Combines routers with Chain for fallback and Merge for averaging.
  • 05Supports hybrid search, multiple vector stores and multi-modal image routes.
  • 06Runs fully locally with FastEmbed without API keys.

Use cases

  • Deciding which tool or agent a chatbot should invoke before calling an LLM.
  • Filtering off-topic or sensitive queries in a customer support assistant.
  • Building low-latency intent classification for hotlines or triage workflows.
  • Routing user messages to different prompts or pipelines based on topic.

Quick start

  1. 1

    Install Semantic Router with the local FastEmbed encoder

    pip install -qU "semantic-router[fastembed]"
  2. 2

    Define routes, create a FastEmbed encoder, and build an IndexedRouter, then call it on a query

Things to know

  • Routing quality depends on the example utterances you write and on calibrating thresholds with labelled data.
  • Some features (OpenAI, Cohere, Pinecone and others) need API keys or external services.

Topics

#llm#routing#semantic-search#embeddings#agents#intent-classification#chatbot#ai#artificial-intelligence#computer-vision

FAQ

What is semantic-router?
A decision layer for LLMs and agents that routes requests by semantic similarity or a small decision model, returning calibrated confidence. Semantic Router decides which route a user request belongs to without waiting for a full LLM generation. Each route is defined by example utterances; incoming queries are embedded and compared against them in vector space, so a decision takes milliseconds. Alternatively, a DecisionRouter asks a small decision model (Jev, Laya or Von) to pick the applicable route in a single call.
Is semantic-router free to use? What license does it have?
semantic-router is open source under the MIT license. Check the license terms before using it in commercial products.
Is semantic-router still maintained?
Actively maintained. The last commit to semantic-router was on Oct 21, 2026.
What can I use semantic-router for?
Deciding which tool or agent a chatbot should invoke before calling an LLM. Filtering off-topic or sensitive queries in a customer support assistant. Building low-latency intent classification for hotlines or triage workflows. Routing user messages to different prompts or pipelines based on topic.
How do I install semantic-router?
Install Semantic Router with the local FastEmbed encoder: pip install -qU "semantic-router[fastembed]" Define routes, create a FastEmbed encoder, and build an IndexedRouter, then call it on a query
Where can I get semantic-router?
The source code and installation instructions are on GitHub: https://github.com/aurelio-labs/semantic-router

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects