Repolode
Submit
theagentrouter

theagentrouter/agent-router

An open source control plane built on Envoy that gives apps one OpenAI-compatible API across hosted LLM providers, self-hosted inference and MCP servers.

View on GitHubWebsite
Stars
2,250
Forks
423
Open issues
368
Last commit
8 hours ago

What is it?

Agent Router is a gateway for AI and agent traffic. Application teams call one OpenAI-compatible API regardless of which model or tool sits behind it, whether that is a hosted provider, a self-hosted inference cluster, or an MCP server. Platform teams manage credentials, routing, quotas, failover and usage attribution in one place, and enforcement is handled by Envoy and Envoy Gateway.

The project was formerly known as Envoy AI Gateway and keeps the same CRDs, API group, CLI name and container images. On a laptop, the aigw CLI can run it as a standalone OpenAI-compatible router on localhost. On Kubernetes, it deploys through Envoy Gateway with custom resources such as AIGatewayRoute and AIServiceBackend.

The project describes a two-tier pattern: a Tier One gateway handles authentication, top-level routing and global rate limiting, while a Tier Two gateway controls access to self-hosted model serving clusters and supports endpoint picker for inference optimization.

Key features

  • 01Exposes one OpenAI-compatible API in front of many model providers.
  • 02Supports hosted providers such as OpenAI, Anthropic, AWS Bedrock and Google Gemini, plus self-hosted inference.
  • 03Manages credentials, routing, quotas and failover centrally.
  • 04Runs standalone on a laptop with the aigw CLI or on Kubernetes via Envoy Gateway.
  • 05Provides a two-tier gateway pattern with endpoint picker support for self-hosted models.

Use cases

  • Giving application teams one API key and endpoint for several LLM providers.
  • Centralizing rate limits, quotas and usage tracking for AI traffic across an organization.
  • Routing requests between hosted models and self-hosted inference clusters on Kubernetes.
  • Running a local OpenAI-compatible endpoint for development with aigw run.

Quick start

  1. 1

    Run Agent Router standalone on your laptop with your OpenAI key

    OPENAI_API_KEY=sk-your-key aigw run
  2. 2

    Point any OpenAI-compatible client at the local endpoint http://localhost:1975/v1

  3. 3

    Read the CLI guide for installation and provider auto-configuration

Things to know

  • The README does not give a package-manager install command; installation is covered in the external CLI guide.
  • Deploying on Kubernetes requires Envoy Gateway and is a separate, more involved setup.

Topics

#llm#api-gateway#envoy#kubernetes#inference#openai-compatible#mcp#ai#ai-gateway#envoyproxy

FAQ

What is agent-router?
An open source control plane built on Envoy that gives apps one OpenAI-compatible API across hosted LLM providers, self-hosted inference and MCP servers. Agent Router is a gateway for AI and agent traffic. Application teams call one OpenAI-compatible API regardless of which model or tool sits behind it, whether that is a hosted provider, a self-hosted inference cluster, or an MCP server. Platform teams manage credentials, routing, quotas, failover and usage attribution in one place, and enforcement is handled by Envoy and Envoy Gateway.
Is agent-router free to use? What license does it have?
agent-router is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is agent-router still maintained?
Actively maintained. The last commit to agent-router was on Oct 20, 2026.
What can I use agent-router for?
Giving application teams one API key and endpoint for several LLM providers. Centralizing rate limits, quotas and usage tracking for AI traffic across an organization. Routing requests between hosted models and self-hosted inference clusters on Kubernetes. Running a local OpenAI-compatible endpoint for development with aigw run.
How do I install agent-router?
Run Agent Router standalone on your laptop with your OpenAI key: OPENAI_API_KEY=sk-your-key aigw run Point any OpenAI-compatible client at the local endpoint http://localhost:1975/v1 Read the CLI guide for installation and provider auto-configuration
Where can I get agent-router?
The source code and installation instructions are on GitHub: https://github.com/theagentrouter/agent-router

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects