Repolode
Submit
pollen-robotics

pollen-robotics/microduck_rl

Reinforcement-learning training environments that teach the Microduck ~25 cm bipedal robot to walk, stand up, and perform tricks in MuJoCo simulation.

View on GitHubWebsite
Stars
2,827
Forks
680
Open issues
61
Last commit
2 days ago

What is it?

microduck_rl holds the reinforcement-learning training environments for Microduck, a roughly 800 g, 25 cm tall bipedal robot. It is built on mjlab (MuJoCo Warp) and trains PPO policies at 50 Hz. Trained policies are exported to ONNX and deployed on the physical robot by the separate Microduck runtime.

The project covers a range of tasks: velocity-tracking walking, fall recovery, standing up, sit-stand transitions, ground picking, ball kicking, forward rolls, and roller-skate gaits. Sim-to-real transfer is handled through the BAM Dynamixel XL330 actuator model, domain randomization on battery voltage and friction, and optional gear-backlash variants of each task.

The repository also includes tooling for exporting ONNX models, running policies in CPU MuJoCo with keyboard control, and publishing policies to the Hugging Face Hub. Training can run locally on a CUDA GPU or on Hugging Face Jobs.

Key features

  • 01Trains walking, standing, fall recovery, and trick policies for the Microduck biped in MuJoCo.
  • 02Models Dynamixel XL330 servos with the BAM actuator model and domain randomization.
  • 03Provides backlash variants that simulate ±1° gear play in every servo joint.
  • 04Exports policies to ONNX with the observation normalizer baked in for deployment.
  • 05Includes keyboard-driven CPU simulation for testing exported policies.
  • 06Publishes policies to the Hugging Face Hub in a format the robot daemon can load.

Use cases

  • Training a custom gait or trick for a Microduck robot.
  • Researching sim-to-real transfer for small servo-driven bipedal robots.
  • Testing exported ONNX policies in simulation before running them on hardware.

Quick start

  1. 1

    Clone the repository

    git clone https://github.com/pollen-robotics/microduck_rl
  2. 2

    Enter the project directory

    cd microduck_rl
  3. 3

    Train the walking policy (needs a CUDA GPU and uv)

    uv run train Mjlab-Velocity-Flat-MicroDuck --env.scene.num-envs 4096
  4. 4

    Drive the exported policy in CPU MuJoCo with the keyboard (after exporting to ONNX)

    uv run scripts/infer_policy.py --walking output.onnx

Things to know

  • Requires a CUDA GPU for training, since training runs through MuJoCo Warp.
  • The 3D model meshes are licensed CC BY-NC-SA 4.0, which is non-commercial, while the code is Apache-2.0.
  • A usable walking gait takes roughly 1-2 hours of training at 4096 envs.

In collections

Reinforcement learning frameworks & training environmentsRobotics & embodied AI projects

Topics

#reinforcement-learning#robotics#mujoco#sim2real#bipedal-robot#ppo#onnx

FAQ

What is microduck_rl?
Reinforcement-learning training environments that teach the Microduck ~25 cm bipedal robot to walk, stand up, and perform tricks in MuJoCo simulation. microduck_rl holds the reinforcement-learning training environments for Microduck, a roughly 800 g, 25 cm tall bipedal robot. It is built on mjlab (MuJoCo Warp) and trains PPO policies at 50 Hz. Trained policies are exported to ONNX and deployed on the physical robot by the separate Microduck runtime.
Is microduck_rl free to use? What license does it have?
microduck_rl is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is microduck_rl still maintained?
Actively maintained. The last commit to microduck_rl was on Nov 5, 2026.
What can I use microduck_rl for?
Training a custom gait or trick for a Microduck robot. Researching sim-to-real transfer for small servo-driven bipedal robots. Testing exported ONNX policies in simulation before running them on hardware.
How do I install microduck_rl?
Clone the repository: git clone https://github.com/pollen-robotics/microduck_rl Enter the project directory: cd microduck_rl Train the walking policy (needs a CUDA GPU and uv): uv run train Mjlab-Velocity-Flat-MicroDuck --env.scene.num-envs 4096 Drive the exported policy in CPU MuJoCo with the keyboard (after exporting to ONNX): uv run scripts/infer_policy.py --walking output.onnx
Where can I get microduck_rl?
The source code and installation instructions are on GitHub: https://github.com/pollen-robotics/microduck_rl

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects