Repolode
Submit
RUC-NLPIR

RUC-NLPIR/FlashRSI

A unified Python framework for running and comparing recursive self-improvement (RSI) methods on AI agent harnesses across 30 benchmarks.

View on GitHubWebsite
Stars
346
Forks
21
Open issues
0
Last commit
yesterday

What is it?

FlashRSI is a research framework for recursive self-improvement of AI agents. It puts 12 RSI algorithms, 30 benchmark suites, and four agent harnesses (MiniAgent, Claude Code, Codex, and Pi) behind one experiment interface, so a method can be swapped or compared without writing a new runner. Experiments are defined in YAML or through the flashrsi CLI, with the RSI method, target harness, worker model, and benchmark chosen independently.

The framework represents a harness as a set of tagged files (prompts, skills, tools, configuration, runtime code) packaged into immutable Snapshots. Each run records the candidate Snapshots, redacted configuration, target and worker trajectories, metrics, and the Snapshot selected at the end. A saved Snapshot can be re-evaluated on other benchmarks with the evaluate command, without running evolution again.

The project is aimed at researchers who want to reproduce RSI results or develop new methods on a shared checkout, execution, and scoring pipeline. It includes a documentation site and guides for adding algorithms, benchmarks, and harnesses.

Key features

  • 01Runs 12 RSI methods, including OPRO, GEPA, EvoPrompt, ACE, and Meta-Harness, through one interface.
  • 02Supports 30 benchmark suites spanning QA, math, coding, web research, documents, and tool use.
  • 03Maps MiniAgent, Claude Code, Codex, and Pi into editable, versioned Snapshots.
  • 04Records candidates, trajectories, metrics, and the selected Snapshot for each run.
  • 05Lets saved Snapshots be evaluated on other benchmarks without re-running evolution.
  • 06Configurable through YAML files or CLI flags with dotted overrides.

Use cases

  • Reproducing published RSI results on agent benchmarks in a shared pipeline.
  • Comparing several self-improvement algorithms on the same harness and benchmark.
  • Testing whether an evolved agent harness transfers to near-domain or out-of-domain tasks.
  • Adding a new RSI method or benchmark to a common experiment framework.

Quick start

  1. 1

    Clone the repository

    git clone https://github.com/RUC-NLPIR/FlashRSI.git
  2. 2

    Enter the directory and create a virtual environment with uv

    cd FlashRSI && uv venv --python 3.12 && source .venv/bin/activate
  3. 3

    Install the framework with all benchmark integrations

    uv sync --extra all
  4. 4

    Set API keys and run a small RSI experiment

    flashrsi run --config examples/quickstart.yaml

Things to know

  • Requires API keys for target and worker models to run experiments.
  • Some benchmarks need Docker, and Harbor tasks need an extra install.
  • Early-stage project with only 70 stars and a Python 3.12 requirement.

In collections

Open-source AI agent frameworks & apps

Topics

#ai-agents#self-improvement#llm#benchmark#prompt-optimization#research-framework#python#agent#recursive-self-improvement

FAQ

What is FlashRSI?
A unified Python framework for running and comparing recursive self-improvement (RSI) methods on AI agent harnesses across 30 benchmarks. FlashRSI is a research framework for recursive self-improvement of AI agents. It puts 12 RSI algorithms, 30 benchmark suites, and four agent harnesses (MiniAgent, Claude Code, Codex, and Pi) behind one experiment interface, so a method can be swapped or compared without writing a new runner. Experiments are defined in YAML or through the flashrsi CLI, with the RSI method, target harness, worker model, and benchmark chosen independently.
Is FlashRSI free to use? What license does it have?
FlashRSI is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is FlashRSI still maintained?
Actively maintained. The last commit to FlashRSI was on Nov 6, 2026.
What can I use FlashRSI for?
Reproducing published RSI results on agent benchmarks in a shared pipeline. Comparing several self-improvement algorithms on the same harness and benchmark. Testing whether an evolved agent harness transfers to near-domain or out-of-domain tasks. Adding a new RSI method or benchmark to a common experiment framework.
How do I install FlashRSI?
Clone the repository: git clone https://github.com/RUC-NLPIR/FlashRSI.git Enter the directory and create a virtual environment with uv: cd FlashRSI && uv venv --python 3.12 && source .venv/bin/activate Install the framework with all benchmark integrations: uv sync --extra all Set API keys and run a small RSI experiment: flashrsi run --config examples/quickstart.yaml
Where can I get FlashRSI?
The source code and installation instructions are on GitHub: https://github.com/RUC-NLPIR/FlashRSI

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects