Repolode
Submit
Denys88

Denys88/rl_games

PyTorch reinforcement learning library (PPO and SAC) built for GPU-vectorized simulators that run thousands of environments in parallel.

View on GitHub
Stars
1,410
Forks
223
Open issues
53
Last commit
23 hours ago

What is it?

rl_games is a reinforcement learning framework implementing PPO and SAC in PyTorch. Its main focus is training on GPU-vectorized simulators, where a single process drives thousands of environments at once. It provides training pipelines for Isaac Lab, MJLab, EnvPool and MyoSuite, along with Gym-style environments such as MuJoCo and Atari.

The library covers features commonly needed for robot learning: an asymmetric actor-critic with a separate central-value critic, RNN policies, multi-GPU training with DistributedDataParallel, population-based training, self-play, and ONNX export of trained policies. Triton-based GAE kernels are used when Triton is installed, and experiments can be logged to Weights and Biases.

Training is configured through YAML files that specify the algorithm, network, and runner settings, and is launched with a runner script or torchrun for multi-GPU runs. The project lists several sim-to-real robotics papers that used it for policy training.

Key features

  • 01PPO and SAC implementations in PyTorch
  • 02Asymmetric actor-critic with a separate central-value critic
  • 03Multi-GPU training via torchrun and DistributedDataParallel
  • 04Population-based training and self-play support
  • 05ONNX export of trained policies
  • 06Training configs via YAML and Weights & Biases logging

Use cases

  • Training locomotion or dexterous manipulation policies in Isaac Lab or MJLab
  • Running high-throughput MuJoCo or Atari experiments with EnvPool
  • Deploying trained RL policies by exporting them to ONNX
  • Scaling RL training across multiple GPUs

Quick start

  1. 1

    Open the Mujoco training example in Google Colab (from the README)

Things to know

  • The MJLab training notebook requires a local GPU, and the Colab version is recommended on L4 or A100 GPUs.
  • The 2.0.0 release is unreleased and changes the API to gymnasium only, requiring Python 3.11 or newer.
  • The legacy Isaac Gym environment is supported only up to rl_games 1.6.5.

In collections

Reinforcement learning frameworks & training environmentsRobotics & embodied AI projects

Topics

#reinforcement-learning#pytorch#gpu-simulation#robotics#ppo#sac#onnx#deep-learning

FAQ

What is rl_games?
PyTorch reinforcement learning library (PPO and SAC) built for GPU-vectorized simulators that run thousands of environments in parallel. rl_games is a reinforcement learning framework implementing PPO and SAC in PyTorch. Its main focus is training on GPU-vectorized simulators, where a single process drives thousands of environments at once. It provides training pipelines for Isaac Lab, MJLab, EnvPool and MyoSuite, along with Gym-style environments such as MuJoCo and Atari.
Is rl_games free to use? What license does it have?
rl_games is open source under the MIT license. Check the license terms before using it in commercial products.
Is rl_games still maintained?
Actively maintained. The last commit to rl_games was on Nov 6, 2026.
What can I use rl_games for?
Training locomotion or dexterous manipulation policies in Isaac Lab or MJLab Running high-throughput MuJoCo or Atari experiments with EnvPool Deploying trained RL policies by exporting them to ONNX Scaling RL training across multiple GPUs
How do I install rl_games?
Open the Mujoco training example in Google Colab (from the README)
Where can I get rl_games?
The source code and installation instructions are on GitHub: https://github.com/Denys88/rl_games

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects