ferro-labs/ai-gateway

What is it?

Key features

• 01Routes requests to 30 LLM providers through a single OpenAI-compatible API.
• 02Offers eight routing strategies, including fallback, load balancing and A/B testing.
• 03Includes built-in guardrails for PII redaction, secret scanning and prompt-injection detection.
• 04Provides response caching, rate limiting and per-key budget controls.
• 05Connects to MCP tool servers over stdio and Streamable HTTP.
• 06Exposes OpenTelemetry tracing and Prometheus metrics, disabled until turned on.

Use cases

• Giving internal teams one endpoint for several LLM providers with failover between them.
• Enforcing PII redaction or content filters before prompts reach external model APIs.
• Tracking token spend and setting per-key budgets across applications.
• Running a self-hosted gateway in Kubernetes or Docker with existing OpenAI SDK clients.

Quick start

1. 1

   Install the ferrogw CLI with npm

   npm install -g ferrogw
2. 2

   Set your upstream provider API key (here OpenAI)

   export OPENAI_API_KEY=sk-your-key
3. 3

   Run ferrogw init to write config.yaml and print your master key; save the key, as it is shown only once

   ferrogw init
4. 4

   Start the server on port 8080 (set GATEWAY_CONFIG and MASTER_KEY first, as described in the README)

   ferrogw serve

Things to know

• !Routing requests requires at least one upstream provider API key, such as OPENAI_API_KEY.

In collections

Topics

FAQ

What is ai-gateway?

Open-source Go gateway that routes LLM requests across 30 providers through one OpenAI-compatible API, with caching, guardrails and cost controls. Ferro Labs AI Gateway is a self-hosted proxy that sits between your application and LLM providers such as OpenAI, Anthropic and AWS Bedrock. Clients send standard OpenAI-format requests to one endpoint, and the gateway forwards them to the configured provider, with routing strategies including fallback, load balancing, least latency, cost-optimized and A/B testing.

Is ai-gateway free to use? What license does it have?

ai-gateway is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.

Is ai-gateway still maintained?

Actively maintained. The last commit to ai-gateway was on Nov 2, 2026.

What can I use ai-gateway for?

Giving internal teams one endpoint for several LLM providers with failover between them. Enforcing PII redaction or content filters before prompts reach external model APIs. Tracking token spend and setting per-key budgets across applications. Running a self-hosted gateway in Kubernetes or Docker with existing OpenAI SDK clients.

How do I install ai-gateway?

Install the ferrogw CLI with npm: npm install -g ferrogw Set your upstream provider API key (here OpenAI): export OPENAI_API_KEY=sk-your-key Run ferrogw init to write config.yaml and print your master key; save the key, as it is shown only once: ferrogw init Start the server on port 8080 (set GATEWAY_CONFIG and MASTER_KEY first, as described in the README): ferrogw serve

Where can I get ai-gateway?

The source code and installation instructions are on GitHub: https://github.com/ferro-labs/ai-gateway