Microduck Lab 🦆: RL experimentation on your Mac

Train reinforcement-learning policies for the Microduck, Pollen Robotics' open-source ~25 cm bipedal robot, on an ordinary Apple Silicon Mac with no CUDA GPU. Watch every policy walk, learn, and backflip live in your browser.

The duck lab viewer: nine ducks running live, mid-backflip and mid-headstand

Running (locally trained on CPU, BAM actuator physics) Backflip showcase: spotter-assisted launch, policy landing, stand handoff
running backflip

The official microduck_rl stack trains through MuJoCo Warp and needs a CUDA GPU. This project runs on the laptop you already have. It's a prototyping loop for reward design, curricula, and new tricks, built on the same MJCF robot model, the same 61-obs / 14-action deployment contract, and the same 50 Hz timing. A behavior you invent here ports straight to the official stack for the final sim2real run, and the ONNX you export is drop-in compatible with the official tooling.

Not affiliated with Pollen Robotics. Two of their repos are used as side-by-side checkouts (see setup).

What's in the box

  • microduck_local/: CPU-MuJoCo + Stable Baselines 3 PPO harness
    • train-walk / train-behavior: velocity-command walking and a library of teachable tricks, with mjlab-distilled rewards, symmetry augmentation, obs normalization, and a penalty-sign guard
    • Two actuator models: fast linearized XML servos, or the honest BAM XL330 voltage model (numba-fused) for maneuvers that saturate servos
    • export-walk: ONNX export with the obs normalizer baked in
    • eval-walk: headless eval battery (falls, command tracking)
    • render-rollout: mp4 for humans, plus a captioned frame contact sheet an AI assistant can read, carrying per-frame heights, angles, and contacts (example)
    • bench-walk / bench-envs: find the right worker count for your machine
    • duck-lab: the streaming backend that drives the browser viewer
  • duck-viewer/: Next.js + react-three-fiber viewer
    • Many ducks side by side, live over WebSocket at 25 Hz; drag policy chips onto ducks to hot-swap brains mid-stride
    • 🎓 Teach panel: ask for one of nine built-in tricks ("stand on one leg") — keyword-matched, no LLM in the loop — see its reward recipe in plain English, watch the trainee improve every ~15 s as live snapshots hot-load, then drag the reward sliders and fine-tune. Reward shaping with no Python in the loop.
    • Staged curricula for hard tricks (the backflip is 5 chained stages), with the viewer narrating the chain
    • 🎬 Animate panel: a keyframe pose editor with a game-style control rig. Author a motion clip in the browser, then "train this" makes RL learn to physically execute it
    • 🎥 Capture panel: 📷 for a full-res PNG, 🎥 to have the camera frame a duck and film it. The lab converts the take to an mp4 and a GIF you can paste straight into a PR
    • ⤓ ONNX download on any run (the baked export, normalizer included), and ⚙ settings to connect your own Hugging Face token (stored and validated today; the GPU-job launcher is not wired up yet)

Teaching a trick from the browser

Quick start

Prereqs: macOS on Apple Silicon (Linux works too), uv, Node 20+, ~3 GB of disk for the checkouts and models.

git clone https://github.com/jonathanhawkins/microduck-lab && cd microduck-lab
git clone https://github.com/pollen-robotics/microduck        # shipped policies, docs
git clone https://github.com/pollen-robotics/microduck_rl     # MJCF models, official stack

cd microduck_local
uv sync
uv run --with pytest pytest tests/        # contract tests, should be all green

# train your first walking policy (a few minutes on an M-series Mac)
uv run train-walk --envs 32 --steps 3_000_000 --run-name first-gait
uv run export-walk runs/first-gait
uv run eval-walk runs/first-gait/policy.onnx

# fire up the lab + viewer
uv run duck-lab runs/first-gait ../microduck/policies/alpha_walking.onnx
cd ../duck-viewer && npm install && npm run dev   # open the printed URL

Then open the 🎓 teach panel and ask the duck to "stand on one leg".

Performance (measured, Apple M-series)

Every number below is reproduced in microduck_local/README.md with its methodology, including the experiments that got rejected.

  • ~16.5k env-steps/s on the shipped 32-env recipe (about 1 min per 1M steps); ~27k steps/s peak in throughput configs
  • One compiled MuJoCo model shared across all workers (fork + copy-on-write): 64 envs dropped from 41 GB to 1.5 GB of memory
  • Semaphore + shared-memory IPC instead of pipes/pickles per step
  • numba-fused BAM actuator kernels, bitwise-identical to the numpy reference
  • Optional MPS (Apple GPU) PPO updates, auto-enabled only where they measured faster
  • Two throughput "wins" (overlapped updates, big-batch) raised steps/s 25-40% but halved learning per step in seed-matched A/Bs, so they ship as opt-in flags rather than defaults. Reward per wall-second is the metric that matters.

Teach the duck your own trick

Tricks are plain-English reward recipes in behaviors/. A Behavior is a set of reward terms, chat keywords, and optionally a staged curriculum for the harder maneuvers. Add one, lock it with a test, and it shows up in the viewer's teach panel with live sliders. The full playbook is in microduck_local/AGENTS.md: contract invariants, reward-design rules, and the verification discipline that keeps you from fooling yourself.

The teach panel only offers tricks that exist in behaviors/: an unrecognized request returns the catalog rather than improvising a recipe. Adding a tenth is a Python change — see Working with AI assistants if you'd rather have a coding agent draft it.

Animate: keyframe a motion, then make it real

The 🎬 animate panel is a pose-and-timeline editor for authoring reference motion clips in the browser, and the bridge from animation to RL.

Dragging the rig's squat handle

  • Pose the duck directly (click a body part, drag) or through the 🎮 control rig, a set of animator-style macro handles (squat, lean, leg swings, sway, stance, twist, toes, look). Each control is a direction in joint space chosen so feet stay planted and the controls are mutually orthogonal: squatting never disturbs the lean slider, and a stride keyed over a crouch keeps the crouch. A ⇕ handle parks on the duck itself, and dragging it down is what drives the squat in the clip above.
  • Keyframe timeline with auto-key, scrub/playback, and looping; the lab solves each pose server-side so the preview duck stays grounded. Clips save to microduck_local/clips/ as plain JSON (run, sprint-cycle, backflip ship as examples).
  • ⚡ Train this: the clip becomes the reward. DeepMimic-style motion imitation ("be in the reference pose for right now") turns an open-ended search like discover a backflip into a tracking problem the policy can solve, without touching the 61-obs deployment contract. See motion.py.

Capture: screenshots, video and GIFs, from the browser

The capture panel sits at the top of the viewer and turns whatever the lab is doing right now into files you can drop into a PR or an issue. No screen recorder, no ffmpeg incantation.

A finished take: the panel offers the mp4 and the gif

  • 📷 shot (always available) downloads a full-resolution PNG of the current view, named after the selected duck, or duck-lab for a crowd shot. The selection ring is hidden for the capture render, and the whole render → read → download runs inside the click's own user gesture, because Chrome silently drops a page's second "automatic" download.
  • 🎥 record (appears once you click a duck) films that one duck for you: the camera glides to a ¾ front shot chosen from its heading, then holds it with a slow drift while MediaRecorder captures the WebGL canvas. Orbit and the camera keys pause for the take, and the footage comes out clean because the labels and panels are DOM, not canvas.
  • ■ stop uploads the take to the lab (POST /captures), whose bundled ffmpeg writes a full-resolution h264 mp4 and a 480 px palette gif into microduck_local/captures/, and the panel offers both as ⬇ downloads. Takes cap at 60 s.

One gotcha: frames are pushed per rendered frame (captureStream(0) + requestFrame()), because automatic capture rides the browser's compositor and records almost nothing in a throttled tab. Keep the tab visible while recording. A take where the scene never rendered gets refused with a message instead of saved as a 0.1 s "video".

Working with AI assistants

This repo is set up for agentic coding tools:

  • AGENTS.md (root and per-project): the workspace map and the training playbook, in the cross-tool convention used by Claude Code, Codex/ChatGPT, and most open-source agents. CLAUDE.md includes it for Claude Code.
  • .claude/skills/: three skills, all of which read as plain documentation for any agent, humans included. render-rollout teaches an agent to look at what a policy actually does (render the rollout, read the contact sheet) before believing reward curves; watch-training does the same for the run that is training right now; restart-servers brings the lab and viewer back up.

Take the brain with you: ⤓ ONNX and 🤗 Hugging Face

⤓ Download the brain. Hover a run in the 🧠 policies panel and a ⤓ appears next to it; one click saves that run's .onnx. You always get policy.onnx, the deployable export with the observation normalizer baked in, and never a raw checkpoint. A checkpoint handed over without its normalizer is quietly a different policy. While a run is still training the button falls back to its newest live.onnx snapshot, so you can pull a brain mid-run. On a staged trick, the chain's ⤓ gives you the final stage: every stage fine-tunes the same network, so the last one is the whole trick.

🤗 Connect Hugging Face (BYOK). The ⚙ button in the duck-lab HUD opens settings, where you paste your own Hugging Face access token. Create one with write access at hf.co/settings/tokens.

The ⚙ settings pane: bring your own Hugging Face key

Bring your own key: your account, your billing, and the token never goes anywhere but huggingface.co. It gets validated with whoami() before anything is written, so a bad paste is rejected rather than stored. It lands in microduck_local/hf-token.json, mode 0600, gitignored. The browser never sees it again: GET /settings/hf returns only your username and a mask like hf_abcd…wxyz. disconnect deletes the file.

That key is for the one step a laptop can't do, retraining a behavior you prototyped here on real GPUs under your own account. Storing and validating the token is what ships today. The HF Jobs launcher isn't wired up yet.

Sim2real, honestly

This harness is for prototyping: minutes-long feedback loops on reward design, observations, and curricula. It runs a subset of the official stack's domain randomization, so don't ship its policies to a real robot. Once a behavior works here, port the env design to an mjlab cfg in microduck_rl and retrain on GPU (that repo's AGENTS.md is the sim2real recipe). Everything here keeps the deployment contract so that port is mechanical.

License

Apache-2.0 (same as the upstream Microduck repos). Not affiliated with or endorsed by Pollen Robotics; "Microduck" is their project.