Repolode
Submit
OpenPipe

OpenPipe/ART

Python RL framework that trains multi-step LLM agents on their own experience using GRPO, with a client-server design that plugs into existing apps.

View on GitHubWebsite
Stars
10,813
Forks
1,008
Open issues
171
Last commit
5 hours ago

What is it?

ART (Agent Reinforcement Trainer) is an open-source reinforcement learning framework for improving LLM agents through their own experience. Developers wrap an existing agentic workflow with the ART client, which collects trajectories of messages and assigns each rollout a reward, then trains the model with GRPO.

The system is split into a client and a server. The client sits in your codebase and exposes an OpenAI-compatible interface. The server runs on any machine with a GPU, handles inference through vLLM and training via LoRA updates, and saves each new checkpoint back into the loop. The client can run on a laptop while the server runs locally or on a remote GPU environment.

The project provides example notebooks for tasks such as email search, 2048, Tic Tac Toe, and Codenames, along with a RULER option for automatic reward generation. It supports vLLM and Hugging Face transformers compatible causal language models, including Qwen and Llama families, and offers a serverless training backend through Weights & Biases.

Key features

  • 01Trains multi-step agents with GRPO on trajectories collected from your own workflow.
  • 02Client-server architecture with an OpenAI-compatible client interface.
  • 03Runs inference on vLLM and updates models through LoRA checkpoints.
  • 04Supports automatic reward generation via RULER.
  • 05Includes example notebooks for games, email search, and SFT distillation.
  • 06Integrates with LangGraph, Langfuse, and Weights & Biases for observability.

Use cases

  • Improving the tool-use reliability of an existing LLM agent without rewriting its code.
  • Training a small open model such as Qwen to perform a narrow multi-step task like email search.
  • Running RL experiments on a laptop client while training on a remote GPU server.
  • Fine-tuning an agent that calls MCP server tools.

Quick start

  1. 1

    Install the package from PyPI

    pip install openpipe-art

Things to know

  • Training needs a GPU, either locally or via the ART server or W&B serverless service.
  • Supported models are limited to vLLM/Hugging Face causal LMs; Gemma 3 is not supported for now.
  • Project describes itself as in active development, so APIs may change.

Topics

#agent#reinforcement-learning#grpo#llm#lora#vllm#python#agentic-ai#llms#qwen

FAQ

What is ART?
Python RL framework that trains multi-step LLM agents on their own experience using GRPO, with a client-server design that plugs into existing apps. ART (Agent Reinforcement Trainer) is an open-source reinforcement learning framework for improving LLM agents through their own experience. Developers wrap an existing agentic workflow with the ART client, which collects trajectories of messages and assigns each rollout a reward, then trains the model with GRPO.
Is ART free to use? What license does it have?
ART is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is ART still maintained?
Actively maintained. The last commit to ART was on Oct 20, 2026.
What can I use ART for?
Improving the tool-use reliability of an existing LLM agent without rewriting its code. Training a small open model such as Qwen to perform a narrow multi-step task like email search. Running RL experiments on a laptop client while training on a remote GPU server. Fine-tuning an agent that calls MCP server tools.
How do I install ART?
Install the package from PyPI: pip install openpipe-art
Where can I get ART?
The source code and installation instructions are on GitHub: https://github.com/OpenPipe/ART

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects