Repolode
Submit
rllm-org

rllm-org/rllm

An open-source framework for training LLM agents with reinforcement learning, supporting any agent harness, sandbox, and training backend.

View on GitHubWebsite
Stars
5,888
Forks
633
Open issues
172
Last commit
4 days ago

What is it?

rLLM is a framework for post-training language agents with reinforcement learning. It lets developers run an existing agent, whether a CLI coding agent like Claude Code or Codex, or a custom agent built with LangGraph or the OpenAI SDK, and then use the collected traces and reward signals to update the model. The same agent code is used for both evaluation and training, so the setup does not need to be rewritten between the two.

Agent rollouts can run in Docker, Daytona, Modal, or locally. Training can use the verl backend for distributed multi-GPU runs, the tinker backend for single-machine training, or the Fireworks platform, and the backend is switched with a single flag. The project also includes integrations for more than 60 benchmarks, such as Terminal-Bench 2.0, SWE-bench, AIME, and GPQA, which can be run with the rllm eval command.

The framework supports training methods including GRPO, REINFORCE, RLOO, SFT, and on-policy distillation. A model gateway captures token IDs and logprobs during rollouts and groups them into episodes, trajectories, and steps for the RL update.

Key features

  • 01Works with CLI harnesses such as Claude Code, Codex, and mini-swe-agent, or custom agents written with @rllm.rollout.
  • 02Runs agent rollouts in Docker, Daytona, Modal, or local sandboxes.
  • 03Switches between verl, tinker, and Fireworks training backends with one flag.
  • 04Provides integrated evaluation for more than 60 benchmarks via rllm eval.
  • 05Implements GRPO, REINFORCE, RLOO, SFT, and on-policy distillation.

Use cases

  • Training a coding agent on benchmarks such as SWE-bench or Terminal-Bench using reinforcement learning.
  • Post-training a math or reasoning model with GRPO on datasets like GSM8K or AIME.
  • Improving an existing LangGraph or OpenAI Agents SDK application through RL without rewriting its code.
  • Running the same agent through both evaluation and multi-GPU training pipelines.

Quick start

  1. 1

    Install rLLM with uv (tinker backend)

    uv pip install "rllm @ git+https://github.com/rllm-org/rllm.git"
  2. 2

    Configure your model provider

    rllm model setup
  3. 3

    Evaluate on a benchmark

    rllm eval gsm8k
  4. 4

    Train with RL

    rllm train gsm8k

Things to know

  • Requires Python 3.11 or newer.
  • Training needs a model provider or backend such as tinker, verl with GPUs, or Fireworks.

Topics

#reinforcement-learning#llm-training#agents#grpo#llm-reasoning#distributed-training#benchmarks#agent-framework#agentic-workflow#coding-agent

FAQ

What is rllm?
An open-source framework for training LLM agents with reinforcement learning, supporting any agent harness, sandbox, and training backend. rLLM is a framework for post-training language agents with reinforcement learning. It lets developers run an existing agent, whether a CLI coding agent like Claude Code or Codex, or a custom agent built with LangGraph or the OpenAI SDK, and then use the collected traces and reward signals to update the model. The same agent code is used for both evaluation and training, so the setup does not need to be rewritten between the two.
Is rllm free to use? What license does it have?
rllm is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is rllm still maintained?
Actively maintained. The last commit to rllm was on Oct 16, 2026.
What can I use rllm for?
Training a coding agent on benchmarks such as SWE-bench or Terminal-Bench using reinforcement learning. Post-training a math or reasoning model with GRPO on datasets like GSM8K or AIME. Improving an existing LangGraph or OpenAI Agents SDK application through RL without rewriting its code. Running the same agent through both evaluation and multi-GPU training pipelines.
How do I install rllm?
Install rLLM with uv (tinker backend): uv pip install "rllm @ git+https://github.com/rllm-org/rllm.git" Configure your model provider: rllm model setup Evaluate on a benchmark: rllm eval gsm8k Train with RL: rllm train gsm8k
Where can I get rllm?
The source code and installation instructions are on GitHub: https://github.com/rllm-org/rllm

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects