Train a Biped Robot With RL: Getting Started With Microduck

Microduck is a 25 cm, 800 g open-source biped robot from Pollen Robotics (Hugging Face) that ships with seven pre-trained moves — walking, sitting, kicking, grabbing with its beak, roller-skating, and standing back up after a fall. What makes it worth your time as a developer isn't the cuteness: the entire software stack is on GitHub under Apache-2.0, and you can train new gaits yourself with reinforcement learning in a physics simulator, then deploy the learned policy straight to the real robot.

Why this matters

Most RL demos stop at the simulator. Microduck closes the loop: train in MuJoCo, export to ONNX, and run the same policy onboard at 50 Hz. It's one of the cheapest concrete introductions to the sim-to-real pipeline you can buy, and the code is fully readable, forkable, and retrainable.

How the stack is organized

Two repos carry the work:

  • pollen-robotics/microduck — the SDK and onboard robot software. SSH in and use robotctl to monitor, configure, and update the robot.
  • pollen-robotics/microduck-rl — the RL training and sim-to-real tooling.

Training runs in MuJoCo Warp (built on NVIDIA's Warp framework) with mjlab, a training framework that reimplements the Isaac Lab API. A usable walking gait trains in roughly one to two hours by simulating 4,096 virtual ducks in parallel on an NVIDIA GPU — or on Hugging Face's own infrastructure.

Core pipeline

The workflow breaks into three stages:

  1. Simulate — thousands of virtual ducks run in parallel in MuJoCo Warp, each episode shaped by a task-specific reward.
  2. Train with PPO — mjlab trains the policy in the simulator; the repo includes 13 task families to start from.
  3. Sim-to-real — export the trained policy and deploy it to the robot's onboard policy loop (50 Hz), where it controls the 15 motors with a camera, LiDAR, and two IMUs.

Starting points

The 7 shipped moves are all published and retrainable, so you can pick one as a baseline instead of training from scratch. Want a first experiment? Fine-tune a walking reward on top of an existing gait, observe the reward curve, then compare simulator and real-robot behavior side by side.

Practical tips

  • An NVIDIA GPU (or HF infra) is effectively required for the parallel training loop — don't expect to train gaits on a laptop CPU.
  • Note the licensing split: the software is Apache-2.0, but the mechanical/electronic design files are licensed non-commercially, so this is an open-source software platform, not open-source hardware.
  • For a lightweight way to poke at the sim, community wrappers like microduck-mcp let you drive the simulated duck from any MCP client.

Resources

Leave a Comment

Scroll to top