Repolode
Submit
respanai

respanai/respan-gateway

A single Rust binary that gives any AI SDK one endpoint to reach many LLM vendors, with MCP tools, traffic control and request logging.

Open-source alternative to:OpenRouter

View on GitHubWebsite
Stars
570
Forks
86
Open issues
28
Last commit
yesterday

What is it?

Respan Gateway sits between your code and AI APIs. You point an existing OpenAI, Anthropic or Gemini SDK at it, and it routes the call to any configured chat model, translating between request formats including streaming, tools, images and reasoning.

Beyond chat, it covers realtime voice, image, speech, transcription, embedding and video APIs across several vendors. It can also attach MCP servers and built-in tool vendors such as web search, browsing and code sandboxes, so a plain chat call can use those tools regardless of client or model.

Traffic handling includes named routes, fallback, retries, cheapest or fastest provider selection and hedged requests. A live log records cost, latency and tokens per call, and traces can be sent to Respan. Configuration is a single TOML file, and the gateway can also run in-process from Python or be embedded in a Rust service.

Key features

  • 01Accepts OpenAI Chat, OpenAI Responses, Anthropic Messages and Gemini API requests and translates them across providers.
  • 02Supports realtime voice, images, speech, transcription, embeddings and video across several vendors.
  • 03Attaches MCP servers and built-in tool vendors so any model can call them inside chat requests.
  • 04Offers named routes, fallback, retries, cheapest or fastest selection and hedged requests.
  • 05Logs cost, latency and tokens for every call and can send traces to Respan.
  • 06Runs as a standalone server, an in-process Python library, or an embeddable Rust component.

Use cases

  • Switching an existing app between LLM vendors without changing its SDK code.
  • Adding fallback and retries so a slow or failing model provider does not break requests.
  • Giving every model access to web search, browsing or code execution tools via MCP.
  • Tracking cost and latency across many AI providers from one request log.

Quick start

  1. 1

    Install the gateway with pip

    pip install respan-gateway
  2. 2

    Set your vendor API key and run the gateway

    export OPENAI_API_KEY=sk-... respan-gateway
  3. 3

    Point your SDK at http://localhost:8080 and call any model

Things to know

  • The README's performance figures are the project's own benchmarks, not independently verified.
  • Client keys are required to serve non-loopback addresses, so public deployment needs a config file.
  • The project is young, with 199 stars, so ecosystem and community support are limited.

In collections

Claude Code tools, skills & pluginsOpen-source AI agent frameworks & appsMCP serversOpenTelemetry & observability platforms

Topics

#llm-gateway#llm#ai-gateway#mcp#openai-compatible#rust#observability#llm-routing#ai-agents#anthropic

FAQ

What is respan-gateway?
A single Rust binary that gives any AI SDK one endpoint to reach many LLM vendors, with MCP tools, traffic control and request logging. Respan Gateway sits between your code and AI APIs. You point an existing OpenAI, Anthropic or Gemini SDK at it, and it routes the call to any configured chat model, translating between request formats including streaming, tools, images and reasoning.
Is respan-gateway free to use? What license does it have?
respan-gateway is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is respan-gateway still maintained?
Actively maintained. The last commit to respan-gateway was on Nov 5, 2026.
What can I use respan-gateway for?
Switching an existing app between LLM vendors without changing its SDK code. Adding fallback and retries so a slow or failing model provider does not break requests. Giving every model access to web search, browsing or code execution tools via MCP. Tracking cost and latency across many AI providers from one request log.
How do I install respan-gateway?
Install the gateway with pip: pip install respan-gateway Set your vendor API key and run the gateway: export OPENAI_API_KEY=sk-... respan-gateway Point your SDK at http://localhost:8080 and call any model
Where can I get respan-gateway?
The source code and installation instructions are on GitHub: https://github.com/respanai/respan-gateway

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects