Repolode
Submit
jeremychone

jeremychone/rust-genai

A Rust library giving one common API for chatting with many AI providers using each provider's native protocol.

View on GitHub
Stars
936
Forks
197
Open issues
42
Last commit
3 days ago

What is it?

genai is a Rust client library for generative AI models. It exposes one chat API that works across providers including OpenAI, Anthropic, Gemini, Ollama (for local models), Groq, xAI, DeepSeek, Cohere, Mistral, and many others. The adapter for each model is chosen from the model name prefix, or forced with a namespace such as `groq::` or `ollama_cloud::`.

The library uses each provider's native protocol where one exists, so provider-specific features such as reasoning controls, thinking budgets, streaming metadata, and multimodal inputs can be represented. Where a provider mainly offers an OpenAI-compatible API, genai uses that layer. Custom OpenAI-compatible endpoints can be configured through environment variables or a ServiceTargetResolver for custom auth and endpoints.

The focus is text chat, vision, and function calling. It is built around ergonomic, common-denominator APIs rather than complete coverage of every provider's SDK. TLS backend is selectable through Cargo features.

Key features

  • 01Unified chat API across many AI providers, including local Ollama models
  • 02Model adapter chosen automatically from the model name or forced with a namespace
  • 03Native protocol support where available, with OpenAI-compatible fallback
  • 04Custom endpoints and auth via environment variables or a target resolver
  • 05Image input and function calling support
  • 06Selectable TLS backends (rustls by default, or native-tls)

Use cases

  • Building a Rust application that switches between OpenAI, Anthropic, and Gemini models without rewriting calls
  • Running local models through Ollama behind the same API as cloud providers
  • Connecting a Rust service to a self-hosted OpenAI-compatible inference server
  • Adding vision or tool-calling features to a Rust backend

Quick start

  1. 1

    Add the crate to Cargo.toml

    cargo add genai
  2. 2

    Create a client and call exec_chat with a model name

Things to know

  • Pre-1.0 release (0.7.x) with breaking changes between minor versions.

In collections

Claude Code tools, skills & pluginsRun LLMs locally

Topics

#llm#rust#multi-provider#ollama#openai#anthropic#gemini#chat-completion

FAQ

What is rust-genai?
A Rust library giving one common API for chatting with many AI providers using each provider's native protocol. genai is a Rust client library for generative AI models. It exposes one chat API that works across providers including OpenAI, Anthropic, Gemini, Ollama (for local models), Groq, xAI, DeepSeek, Cohere, Mistral, and many others. The adapter for each model is chosen from the model name prefix, or forced with a namespace such as `groq::` or `ollama_cloud::`.
Is rust-genai free to use? What license does it have?
rust-genai is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is rust-genai still maintained?
Actively maintained. The last commit to rust-genai was on Nov 4, 2026.
What can I use rust-genai for?
Building a Rust application that switches between OpenAI, Anthropic, and Gemini models without rewriting calls Running local models through Ollama behind the same API as cloud providers Connecting a Rust service to a self-hosted OpenAI-compatible inference server Adding vision or tool-calling features to a Rust backend
How do I install rust-genai?
Add the crate to Cargo.toml: cargo add genai Create a client and call exec_chat with a model name
Where can I get rust-genai?
The source code and installation instructions are on GitHub: https://github.com/jeremychone/rust-genai

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects