Repolode
Submit
sail-sg

sail-sg/envpool

A C++ batched environment pool for reinforcement learning that runs many Gym-style environments in parallel with a thread pool.

View on GitHubWebsite
Stars
1,533
Forks
147
Open issues
16
Last commit
16 days ago

What is it?

EnvPool is a parallel execution engine for reinforcement learning environments. It manages a pool of environments in C++ with a thread pool and exposes them to Python through pybind11, so one call can step many environments at once. It supports both Gymnasium and dm_env style APIs, in synchronous and asynchronous modes, with batched interaction by default.

The project covers a wide range of environment families, including Atari, MuJoCo, DeepMind Control Suite, Box2D, ViZDoom, Procgen, Minigrid, MPE2, MetaWorld, PGX and Jumanji. Environments can also be added through a C++ developer interface. Rendering to batched RGB arrays and XLA support with JAX jit are available. Prebuilt wheels are published on PyPI for Linux, macOS and Windows.

The benchmarks in the repository compare throughput against Python subprocess-based vector environments and for-loops on machines from 12 to 256 CPU cores.

Key features

  • 01Batched environment stepping through a C++ thread pool with pybind11 bindings.
  • 02Compatible with Gymnasium and dm_env APIs.
  • 03Supports synchronous and asynchronous execution, including single and multi-player environments.
  • 04Covers many environment families such as Atari, MuJoCo, DeepMind Control and Procgen.
  • 05Batched rgb_array rendering and XLA support with JAX jit.

Use cases

  • Speeding up data collection for reinforcement learning training on multi-core machines.
  • Running Atari or MuJoCo experiments with libraries such as Stable-Baselines3, Tianshou or CleanRL.
  • Adding a custom environment with a high-throughput C++ implementation.

Quick start

  1. 1

    Install EnvPool from PyPI

    pip install envpool
  2. 2

    Open a Python console and import envpool to confirm the install worked

Things to know

  • Linux MuJoCo rendering needs the system EGL/OpenGL runtime, such as libegl1, libopengl0 and libgl1-mesa-dri on Ubuntu.
  • On Linux, Procgen environments need a separately installed system Qt 5 runtime, and building from source also requires Qt 5.

In collections

Reinforcement learning frameworks & training environmentsRobotics & embodied AI projects

Topics

#reinforcement-learning#parallel-processing#gym#simulation#cpp#python-bindings#vectorized-environment#atari-games#box2d#cpp17

FAQ

What is envpool?
A C++ batched environment pool for reinforcement learning that runs many Gym-style environments in parallel with a thread pool. EnvPool is a parallel execution engine for reinforcement learning environments. It manages a pool of environments in C++ with a thread pool and exposes them to Python through pybind11, so one call can step many environments at once. It supports both Gymnasium and dm_env style APIs, in synchronous and asynchronous modes, with batched interaction by default.
Is envpool free to use? What license does it have?
envpool is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is envpool still maintained?
Actively maintained. The last commit to envpool was on Oct 21, 2026.
What can I use envpool for?
Speeding up data collection for reinforcement learning training on multi-core machines. Running Atari or MuJoCo experiments with libraries such as Stable-Baselines3, Tianshou or CleanRL. Adding a custom environment with a high-throughput C++ implementation.
How do I install envpool?
Install EnvPool from PyPI: pip install envpool Open a Python console and import envpool to confirm the install worked
Where can I get envpool?
The source code and installation instructions are on GitHub: https://github.com/sail-sg/envpool

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects