sierra-research/tau2-bench

What is it?

Key features

• 01Simulates customer-service domains with defined policies, tools and tasks.
• 02Supports text turn-based evaluation and full-duplex voice evaluation.
• 03Includes airline, retail, telecom, banking knowledge and intake domains.
• 04Provides language packs for six languages across text and voice.
• 05Runs through a `tau2` CLI with dry-run planning and result viewing.

Use cases

• Comparing LLM models or agent designs on realistic customer-service tasks.
• Testing whether an agent follows business policies and calls tools correctly.
• Evaluating voice agents for multilingual customer-service calls.
• Building custom agents and domains for reinforcement learning or research experiments.

Quick start

1. 1

   Clone the repository

   git clone https://github.com/sierra-research/tau2-bench
2. 2

   Enter the project folder and install dependencies with uv

   cd tau2-bench && uv sync
3. 3

   Create the .env file and add your API keys

   cp .env.example .env
4. 4

   Run a small evaluation on the airline domain

   tau2 run --domain airline --agent-llm gpt-4.1 --user-llm gpt-4.1 --num-trials 1 --num-tasks 5

Things to know

• !Running evaluations requires API keys from paid LLM providers.
• !Python 3.12 or newer, below 3.14, is required.
• !Voice features need extra system dependencies and voice IDs you must create yourself.

Topics

FAQ

What is tau2-bench?

A simulation benchmark for evaluating customer-service AI agents that use tools and talk with simulated users in text and voice. τ-Bench (tau2-bench) is a simulation framework for testing how well LLM agents handle customer-service work. Each domain defines a policy the agent must follow, a set of tools it can call, and tasks used to score its performance. The project ships with airline, retail, telecom, banking knowledge retrieval, and entity-capture (intake) domains, plus a mock domain for testing.

Is tau2-bench free to use? What license does it have?

tau2-bench is open source under the MIT license. Check the license terms before using it in commercial products.

Is tau2-bench still maintained?

Actively maintained. The last commit to tau2-bench was on Nov 5, 2026.

What can I use tau2-bench for?

Comparing LLM models or agent designs on realistic customer-service tasks. Testing whether an agent follows business policies and calls tools correctly. Evaluating voice agents for multilingual customer-service calls. Building custom agents and domains for reinforcement learning or research experiments.

How do I install tau2-bench?

Clone the repository: git clone https://github.com/sierra-research/tau2-bench Enter the project folder and install dependencies with uv: cd tau2-bench && uv sync Create the .env file and add your API keys: cp .env.example .env Run a small evaluation on the airline domain: tau2 run --domain airline --agent-llm gpt-4.1 --user-llm gpt-4.1 --num-trials 1 --num-tasks 5

Where can I get tau2-bench?

The source code and installation instructions are on GitHub: https://github.com/sierra-research/tau2-bench