Most of what a teleop operator asks of the legs is small: ten centimetres closer to the counter, a tenth of a radian to face the shelf. We held one small command for ten seconds on 71 configurations of 43 publicly released Unitree G1 walking policies, in MuJoCo and PhysX, and scored movement along the command; 34 are stable enough to score. gr00t_wbc at its canonical 0.74 m comes first in both engines, 37% error against 100% for standing still, yet ignores anything under 0.05. About half the field does not walk until 0.13–0.2 m/s; below 0.05, the few that respond creep or wander.
All 34 scored configurations as a table · and how to read the names
A suffix like _h066_p012 is the commanded torso posture of one network: height 0.66 m, pitch 0.12 rad. _sonicplnr trackers follow sonic’s planner; _openwbtplnr and _gr00twbcplnr follow a planner policy. The two _with_arms rows control all 29 joints. Legs are at native stiffness except where marked.
- error: 0% perfect, 100% standing still, over 100% worse than standing still.
- right way: share of runs moving at least 0.025 along the command.
- delivered: median movement along the command divided by the command.
- drift at zero: median speed when asked for under 0.02 m/s, m/s, MuJoCo / PhysX.
| policy | error | error MuJoCo / PhysX | walk: right way | walk: delivered | drift at zero m/s | turn: right way | turn: delivered |
|---|---|---|---|---|---|---|---|
| gr00t_wbc 0.74 m | 37% | 41 / 35% | 97% | 0.85 | 0.001 / 0.002 | 99% | 0.90 |
| supermimic_dyn gr00t_wbc planner policy | 45% | 46 / 45% | 88% | 0.71 | 0.007 / 0.007 | 98% | 1.00 |
| morphmimic gr00t_wbc planner policy | 54% | 57 / 53% | 61% | 0.39 | 0.006 / 0.005 | 100% | 0.99 |
| openwbt | 62% | 53 / 69% | 90% | 0.79 | 0.042 / 0.053 | 99% | 0.75 |
| morphmimic openwbt planner policy | 63% | 58 / 69% | 69% | 0.49 | 0.023 / 0.050 | 99% | 0.78 |
| gr00t_wbc 0.70 m | 65% | 67 / 64% | 80% | 0.58 | 0.001 / 0.002 | 86% | 0.59 |
| mimic_lite openwbt planner policy | 67% | 67 / 66% | 42% | 0.18 | 0.013 / 0.008 | 99% | 0.80 |
| supermimic_dyn openwbt planner policy | 68% | 65 / 72% | 85% | 0.75 | 0.042 / 0.050 | 97% | 0.76 |
| gr00t_wbc 0.66 m, 0.12 rad, legs ×1.5 | 69% | 70 / 67% | 85% | 0.53 | 0.001 / 0.001 | 83% | 0.60 |
| supermimic_panhainan gr00t_wbc planner policy | 70% | 69 / 71% | 41% | 0.15 | 0.013 / 0.023 | 99% | 0.98 |
| wbc_agile_velocity | 75% | 72 / 76% | 11% | 0.02 | 0.002 / 0.001 | 69% | 0.54 |
| gr00t_wbc 0.66 m, legs ×1.5 | 77% | 77 / 74% | 77% | 0.53 | 0.001 / 0.001 | 81% | 0.59 |
| mimic_lite gr00t_wbc planner policy | 78% | 79 / 76% | 41% | 0.09 | 0.021 / 0.019 | 100% | 1.05 |
| zealot | 85% | 85 / 96% | 50% | 0.36 | 0.011 / 0.001 | 21% | 0.09 |
| wbc_agile | 88% | 87 / 89% | 37% | 0.11 | 0.003 / 0.003 | 16% | 0.17 |
| sonic | 93% | 91 / 94% | 4% | 0.01 | 0.006 / 0.007 | 44% | 0.22 |
| mimic_lite follows sonic’s planner | 96% | 95 / 98% | 11% | 0.01 | 0.008 / 0.005 | 49% | 0.27 |
| morphmimic follows sonic’s planner | 96% | 94 / 100% | 18% | 0.02 | 0.006 / 0.013 | 39% | 0.18 |
| clobot all 29 joints | 97% | 96 / 98% | 17% | 0.00 | 0.001 / 0.001 | 28% | 0.07 |
| htd_wbc 0.70 m | 99% | 100 / 86% | 26% | 0.01 | 0.001 / 0.010 | 24% | 0.03 |
| handoff all 29 joints | 99% | 95 / 99% | 48% | 0.01 | 0.000 / 0.000 | 51% | 0.52 |
| htd_wbc 0.74 m | 100% | 100 / 99% | 11% | 0.00 | 0.001 / 0.001 | 1% | 0.01 |
| homie | 100% | 99 / 101% | 14% | 0.00 | 0.001 / 0.001 | 0% | 0.01 |
| rl_lab | 100% | 99 / 100% | 24% | 0.01 | 0.001 / 0.002 | 0% | 0.01 |
| htd_wbc 0.66 m, 0.12 rad | 100% | 100 / 139% | 29% | 0.00 | 0.000 / 0.124 | 25% | 0.02 |
| amo | 100% | 104 / 96% | 44% | 0.05 | 0.002 / 0.005 | 77% | 1.07 |
| robomimic | 101% | 100 / 101% | 13% | 0.00 | 0.004 / 0.004 | 20% | 0.08 |
| htd_wbc 0.66 m | 101% | 100 / 180% | 32% | 0.00 | 0.001 / 0.153 | 27% | 0.01 |
| supermimic_dyn follows sonic’s planner | 103% | 101 / 106% | 41% | 0.12 | 0.013 / 0.023 | 44% | 0.20 |
| supermimic_rayzhao openwbt planner policy | 105% | 100 / 111% | 67% | 0.48 | 0.049 / 0.062 | 75% | 0.65 |
| supermimic_rayzhao follows sonic’s planner | 106% | 104 / 108% | 29% | 0.05 | 0.027 / 0.033 | 42% | 0.17 |
| supermimic_rayzhao gr00t_wbc planner policy | 111% | 110 / 112% | 49% | 0.24 | 0.078 / 0.073 | 75% | 0.76 |
| falcon | 135% | 141 / 125% | 69% | 0.82 | 0.132 / 0.112 | 100% | 0.95 |
| asap | 172% | 172 / 170% | 69% | 0.75 | 0.129 / 0.117 | 66% | 0.52 |
The two gr00t_wbc 0.66 m rows at native stiffness complete 93.6/93.7% of walking runs and score 108/104%, moving the right way 37–39% of the time.
What you can reuse
- The test: one command held ten seconds, scored along the command rather than by speed, in two physics engines.
- The data: 4096 runs per test, engine and leg stiffness for all 71 configurations, plus a measured-against-commanded chart per scored policy. Code + data, MIT; checkpoints under their authors’ terms. github.com/rhoyn/teleop-walking-bench
- Where small commands get thrown away, with file and line.
1Small commands are most of teleop, and a policy can drop them three ways
Every approach ends with the last ten centimetres. A deadzone ignores the stick until it is pushed far enough, so the operator overshoots and corrects. Creep moves the robot while the hands manipulate. Slide translates the camera when only a turn was asked, and that motion lands in the egocentric video and its action labels (see low-latency eyes).
Each run holds one command for ten seconds after a three-second crane hold, no punches: a walk in any direction up to 1 m/s, or a turn up to 1 rad/s. Only commands of 0.05 to 0.15 are scored. Error is the gap between measured and commanded velocity over the command; turn error adds any planar speed. Walk and turn are averaged, both engines pooled.Full protocol · sampling, scoring, acceptance
2gr00t_wbc at 0.74 m follows small commands best
gr00t_wbc_h074_p000 scores 37%, and first in each engine: 41% in MuJoCo, 35% in PhysX. It moves the right way in 97% of small walks and 99% of small turns, delivering 0.85 and 0.90 of the command. The next policy that walks on its own, openwbt, is about 25 points behind, far beyond the intervals. It is also the pick of our punch benchmark, where the leaders tie.
The engines order the next rows differently: openwbt scores 53/69%, so in PhysX gr00t_wbc at 0.70 m (67/64%) beats it. The 0.66 m rows are wrong for teleop: they end punched PhysX segments 122–147 cm from target.
3Its limits: a 0.05 deadzone, a weak left sidestep, and an engine gap
- A sharp deadzone at 0.05. Turning at 0.02–0.05 rad/s gives 0.002/0.000; at 0.05–0.10, 0.072/0.058. Walking is the same. Below a command norm of 0.05 GR00T hands over to its balance network (
policy.cpplines 8, 93), so corrections under 5 cm/s or 0.05 rad/s do nothing. - Almost still at zero. 1–2 mm/s of drift, not none.
- Little slide. Turning on the spot, it translates at 2.0/2.3 cm/s, against 4.4/5.8 for openwbt and 10–13 for falcon and asap.
- An engine gap. Its MuJoCo–PhysX yaw-rate gap, 0.093, is among the field’s largest (average 0.036): near 1 rad/s it turns at 0.97 in MuJoCo, 0.80 in PhysX. A sim-to-real caution.
4Two trackers come second and third by copying gr00t_wbc
13 of the 34 scored are motion trackers. Of the five scored trackers that follow the gr00t_wbc planner policy, two come second and third: supermimic_dyn at 45% [44–47] and morphmimic at 54%. Their reference comes from gr00t_wbc at 0.74 m; the other three score 70–111%. supermimic_dyn completes only 20/19% of punched tours._sonicplnr follow sonic’s planner; _openwbtplnr and _gr00twbcplnr follow a planner policy. Any walking you see is slide, scored as error.
5openwbt creeps below 0.05; falcon and asap wander
openwbt creeps forward at 4–6 cm/s, even when asked for under 2 cm/s. Its code stops stepping only when all three commands are exactly zero (policy.cpp line 71). Under punches it survives about a third as long as gr00t_wbc.
falcon and asap only look responsive. Asked for under 2 cm/s they walk off, mostly backwards, at 11–13 cm/s. A fifth to a quarter of small walk requests send them backwards. falcon always turns the right way (delivered 0.95): its 138% turn error is walking off, 8% without it.
6Common training code ignores 0.2 m/s
legged_gym and unitree_rl_gym zero training commands at or under 0.2 m/s, so robots never practise walking slowly; OpenHomie penalises stepping under 0.1. homie, rl_lab, htd_wbc, sonic and wbc_agile_velocity don’t walk until about 0.13–0.20 m/s: htd_wbc at 0.74 m in MuJoCo gives 0.000 at 0.05–0.15, then 0.178 at 0.20–0.25. homie, rl_lab and htd_wbc at 0.74 m start turning only at 0.39–0.65 rad/s.
homie deploys with a yaw scale of 0.25 where training used 0.5, on a training line marked TODO change this; its turn ratio of 0.43–0.50× above onset is consistent with that 2× mismatch (not rerun).
7On our own G1, 0.2 rad/s gave zero
One test ran on hardware, with gr00t_wbc crouched at 0.66 m, not the 0.74 m pick. Raising a turn request slowly from nothing to 0.8 rad/s, it turned at roughly a fifth of what we asked. Held at 0.2 rad/s it did not turn at all; at 0.4 it crept round at 0.01; at 0.6 it managed 0.1. It shuffled, lifting its feet less than a centimetre: asked to swing an ankle half a radian, the joint moved 0.03. A policy that expects firmer joints than the real motors give drags its feet.
8Stiffer legs do not fix the pick
Scaling leg stiffness and damping by 1.5 or 2 helps crouched gr00t_wbc: the 0.66 m rows start turning at 0.07 rad/s instead of 0.35–0.41, in MuJoCo only (PhysX: 0.09 at ×1 and ×1.5, 0.07 at ×2). It makes the 0.74 m pick worse: 37%, 53%, 66%. Under punches 27 of 71 configurations fall sooner at 1.5×, 43 at 2×.
9Pick by job
- Final approach and fine placement: gr00t_wbc at 0.74 m. Mind the 0.05 deadzone and the weak left sidestep. It is under the NVIDIA Open Model licence, commercial use permitted with attribution.
- Corrections under 0.05: openwbt, if you cancel its forward creep and keep it away from contact.
- Avoid for slow work: htd_wbc, homie, rl_lab, sonic and wbc_agile_velocity (0.13–0.2 m/s deadzone), falcon and asap (they wander backwards), and the 0.66 m rows.
10Limitations
- Everything is in simulation, MuJoCo and PhysX, except the robot test.
- This article measures how publicly released walking policies follow small commands in simulation; it doesn’t explain any stack’s internals.
11Measure your own stack
The campaign needs an RTX-class NVIDIA GPU; the charts only need the result files.
| step | command |
|---|---|
| build | ./download_export_onnx.sh && make && make capture |
| turn test | ./benchmark.sh --benchmark rotate |
| walk test | ./benchmark.sh --benchmark direction |
| results | results/rotate_gainx<G>/r<NN>.<policy>.<engine>.csv, d<NN> in direction_gainx<G>/, one line per run id |
| charts + table | make cmd_vs_real_movements && ./build/cmd_vs_real_movements |
Each CSV line holds command and measured velocity, so any scoring can be recomputed. Read error first, then right way, which tells a deadzone from a wander, then drift at zero per engine.
12Try it yourself
This is gr00t_wbc at 0.74 m, running at 50 Hz against MuJoCo in your browser. Click the robot, then W/S walks forward and back, A/D steps sideways and Q/E turns. Pick how big each command is, and try 0.05: