Repolode
Submit
yym68686

yym68686/uni-api

Self-hosted gateway that exposes many LLM providers (OpenAI, Anthropic, Gemini, Vertex, Azure, AWS and more) behind one OpenAI-compatible API with load balancing.

Open-source alternative to:OpenRouter

View on GitHubWebsite
Stars
1,270
Forks
158
Open issues
15
Last commit
3 days ago

What is it?

uni-api is a self-hosted gateway for large language model APIs. It lets you call multiple backend providers, including OpenAI, Anthropic, Gemini, Vertex, Azure, AWS, xAI, Cohere, Groq, Cloudflare and OpenRouter, through a single endpoint that speaks the OpenAI format. Providers and their keys are defined in a YAML configuration file rather than a web console, and models can be renamed or grouped so clients use simple names.

The gateway handles operational concerns that come with running several API keys and providers. It supports weighted, round-robin and other scheduling strategies across channels and across keys within a channel, retries the next channel on failure, and cools down failing channels for a set period. Per-key permissions with wildcards, rate limits, model timeouts and optional message moderation are configured in the same file.

Besides chat completions, it exposes OpenAI-style routes for responses, images, embeddings, audio transcription and speech, moderations and model listing. It is written in Rust and can be run from a Docker image, with SQLite by default or PostgreSQL as the database.

Key features

  • 01Exposes multiple providers through one OpenAI-compatible API, including OpenAI, Anthropic, Gemini, Vertex and Azure.
  • 02Configured entirely through a YAML file, with no web frontend required.
  • 03Channel-level weighted and round-robin load balancing, plus key-level rotation within a channel.
  • 04Automatic retry on the next channel and cooldown of failing channels.
  • 05Per-key model permissions with wildcards, rate limits and per-model timeouts.
  • 06Supports chat, responses, images, embeddings, audio, moderations and model listing endpoints.

Use cases

  • Running a personal or small-team API endpoint that routes to several LLM providers with one key.
  • Spreading traffic across multiple API keys or accounts to reduce rate-limit errors.
  • Giving existing OpenAI-compatible clients access to Claude or Gemini models without code changes.
  • Hosting a shared gateway with per-user keys, quotas and moderation.

Quick start

  1. 1

    Write an api.yaml configuration file with your provider base_url and API key

  2. 2

    Start uni-api with the configuration file, mounted as api.yaml or supplied via CONFIG_URL, as described in the README

Things to know

  • You must write a YAML configuration file before uni-api can start.

In collections

Claude Code tools, skills & pluginsSelf-hosted appsMicroservice frameworks & service infrastructure

Topics

#llm-gateway#openai-compatible#load-balancing#api-gateway#anthropic#gemini#self-hosted#rust#aws#azure

FAQ

What is uni-api?
Self-hosted gateway that exposes many LLM providers (OpenAI, Anthropic, Gemini, Vertex, Azure, AWS and more) behind one OpenAI-compatible API with load balancing. uni-api is a self-hosted gateway for large language model APIs. It lets you call multiple backend providers, including OpenAI, Anthropic, Gemini, Vertex, Azure, AWS, xAI, Cohere, Groq, Cloudflare and OpenRouter, through a single endpoint that speaks the OpenAI format. Providers and their keys are defined in a YAML configuration file rather than a web console, and models can be renamed or grouped so clients use simple names.
Is uni-api free to use? What license does it have?
uni-api is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is uni-api still maintained?
Actively maintained. The last commit to uni-api was on Nov 4, 2026.
What can I use uni-api for?
Running a personal or small-team API endpoint that routes to several LLM providers with one key. Spreading traffic across multiple API keys or accounts to reduce rate-limit errors. Giving existing OpenAI-compatible clients access to Claude or Gemini models without code changes. Hosting a shared gateway with per-user keys, quotas and moderation.
How do I install uni-api?
Write an api.yaml configuration file with your provider base_url and API key Start uni-api with the configuration file, mounted as api.yaml or supplied via CONFIG_URL, as described in the README
Where can I get uni-api?
The source code and installation instructions are on GitHub: https://github.com/yym68686/uni-api

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects