YouDuck.ai
≡

Set up training and get your first policy

Last updated

Microduck’s motion policies are not trained in the robot’s repository but next door, in microduck_rl. This page is the shortest path from nothing to one finished training run.

What you need

  • A CUDA GPU — training runs on MuJoCo Warp.
  • uv — Python environment management.

Underneath is mjlab (MuJoCo Warp) plus PPO. Policies are trained here at 50 Hz, exported to ONNX, and loaded by the runtime in the robot repository — the training rate and the onboard control loop are the same 50 Hz, and that is not a coincidence.

Four commands

git clone https://github.com/pollen-robotics/microduck_rl
cd microduck_rl

Train a walking policy (on your GPU; at 4096 parallel environments a usable gait takes roughly 1–2 hours):

uv run train Mjlab-Velocity-Flat-MicroDuck --env.scene.num-envs 4096

Watch the trained policy in the viewer:

uv run play Mjlab-Velocity-Flat-MicroDuck --wandb-run-path <entity/project/run_id>

Export to ONNX for deployment:

uv run scripts/export.py Mjlab-Velocity-Flat-MicroDuck --wandb-run-path <...>

Drive the exported policy from the keyboard in CPU MuJoCo:

uv run scripts/infer_policy.py --walking output.onnx

Resuming from a checkpoint

uv run train Mjlab-Velocity-Flat-MicroDuck --env.scene.num-envs 4096 \
    --agent.run-name resume --agent.load-checkpoint model_29999.pt --agent.resume True

No GPU

Add --hf-jobs to any train command and the run happens on Hugging Face Jobs instead of locally. See scripts/hf/README.md in the repository.

Rehearsing real policy switching in simulation

On the robot the runtime hot-swaps between these policies (walk / recover / trick), backed by a shared 61-dimensional observation contract, so any one of them can take the robot over at any moment.

scripts/infer_policy.py rehearses exactly that:

uv run scripts/infer_policy.py --walking walk.onnx --standing stand.onnx \
    --sitstand sitstand.onnx --roulade roulade.onnx --new-cmd-obs

Keyboard: velocity commands, G to crouch and pick, Y to sit and stand, R for the forward roll, K / L to kick left and right.

--debug, --save-csv and --record are there for sim2real comparison.

Putting your own policy on the robot

No release required. Point at it directly in the robot’s /etc/robot/robotd.toml:

[policy]
walk = "/home/radxa/my_walking.onnx"
stand = "/home/radxa/my_stand.onnx"
sudo systemctl restart robotd

Those paths survive system updates — a release replaces the binaries and policy files it ships with, not the config that points elsewhere. Delete the lines to go back to the shipped policies.

A policy that fails to load reports unhealthy, and both robotctl health and the bottom border of robotctl monitor name the reason.