Walk under disturbances: 43 publicly released Unitree G1 walking policies, none finishes reliably

What you can reuse

• The harness: forty-three G1 policies behind one C++ interface, in two physics engines, as seventy-one rows.
• The task: eighteen waypoints, one punch per segment, and arms the policy may not use for balance.
• The data: completion, error and cost per row and engine, plus small-command and leg-stiffness sweeps. Code + data, MIT; checkpoints under their authors’ terms. github.com/rhoyn/teleop-walking-bench
• Twenty-three of the policies, running in your browser.

1Nothing completes reliably, and teleop pays for every fall

In teleop the punches are the operator’s own: a reach that shifts the robot’s weight, a door that pushes back, a bump into a table. The arms are the operator’s, so the policy must balance without them, and a fall ends the episode. Here every segment lands a punch and the harness owns the arms: handoff completes 0/0% of its tours under that rule, 35/36% when allowed its arms.

Short of falling, a robot can drift off target, more the longer it walks; have a deadzone, where small stick inputs do nothing, so the operator pushes harder and overshoots; or creep when told to stand still, while the operator manipulates.

2Which one to pick for teleop: gr00t_wbc at 0.74 m

On completion alone the leaders are a tie. htd_wbc_h066_p012 completes 0.4 points fewer tours than gr00t_wbc_h070_p000, 95% interval −1.7 to +0.8 over both engines, and ranks three to six overlap; the two 0.74 m rows sit a few points lower. So we pick on teleop needs: small commands, standstill drift, position accuracy.

gr00t_wbc at its canonical 0.74 m is the one that does all three. gr00t_wbc_h074_p000 completes 74/77% of tours and ends segments 11/18 cm and 4/4° from target. Its small-command error, 37%, is the best in the field, it drifts 0.001/0.002 m/s when asked to stand still, and 1.5× stiffer legs cost it 1% of its survival (9% at 2×). htd_wbc lands as close or closer, 7–19 cm against 11/18 cm, but has a deadzone on small commands (section 7).

At 0.70 m it gains about 9/7 points of completion, but small-command error rises from 37% to 65%. The 0.66 m rows are wrong for teleop: gr00t_wbc there ends PhysX segments 122–147 cm from target, and htd_wbc creeps at 0.124 and 0.153 m/s in PhysX when it should stand still.

gr00t_wbc is under the NVIDIA Open Model licence, commercial use permitted with attribution; htd_wbc is BSD-3-Clause.

3How the benchmark works

• Tour. Three minutes, eighteen waypoints ten seconds apart, 50 Hz control over a 2 ms step.
• Steering. A P-controller on position and yaw error, by default gain 1.5 with reached bands 10/20 cm and 0.05/0.12 rad (seven policies set their own). Commands are clamped to each policy’s own limits (0.3 m/s for gr00t_wbc and htd_wbc, 1.5 m/s for run_residual, no yaw for amo and dm_march), so the task differs per row.
• Punches. 0.1 s after each waypoint switch: a random one of all 29 joints, a random 3D direction, up to 40 N·s on a 35.1 kg robot, about 1.1 m/s.
• Falls. Pelvis below 0.20 m. Lenient: kneeling or sitting still counts as surviving.
• Porting. Authors’ gains, scales and clocks, at 50 Hz through TensorRT (TF32). The policy owns the fifteen leg and waist joints; the fourteen arm joints random-walk.
• Statistics. Every policy and engine gets the same tours. Wilson 95% intervals are ±1.3–1.9 points at n = 2560 and up to ±4 at n = 512, so the gaps between Figure 2’s groups are not noise.

4The top eight are two policies in four postures

htd_wbc’s authors reviewed our adapter and suggested sweeping torso height and pitch (issue #1); we ran gr00t_wbc over the same four. Only height moves the score. For gr00t_wbc, 0.74 to 0.66 m buys fourteen points of MuJoCo completion for fourteen to eighteen centimetres of position error, and in PhysX the last step gives 10–12 points back. htd_wbc pays nothing in MuJoCo: 79/77% to 86/81%, error down from 12 to 7–8 cm.

5Completing is not going where you were sent

morphmimic_sonicplnr completes 84% in PhysX, level with the best, but ends segments 59/41 cm from target; gr00t_wbc at 0.70–0.74 m and every htd_wbc row land within 7–27 cm. Low error is not enough: wty_cpp ends 28/33 cm off, close to wbc_agile (18/33 cm) and amo (18/49 cm), yet completes 1/1%; falcon almost none (0.2/0.4%). Error is taken where a segment ends or the robot fell, so falling early flatters it.

Costs matter for the data. wbc_agile_velocity stomps hardest among the leaders, 4.5–4.9 J a touchdown, and htd_wbc vibrates two to three times as much as gr00t_wbc in PhysX.

6A tracker is only as good as its reference

mimic_lite, MorphMimic and eight SuperMimic reproductions follow a reference motion but ship no planner. _sonicplnr rows take it from sonic’s planner network; _openwbtplnr and _gr00twbcplnr from a planner policy, a second, unpunched MuJoCo G1 driven by openwbt or gr00t_wbc_h074_p000 that walks 15 ticks (0.3 s at 50 Hz) ahead.

morphmimic completes 4/5% with the OpenWBT planner policy, 62/84% with sonic’s planner and 68/81% with the GR00T WBC one. On a ninety-second tour morphmimic_sonicplnr did 71/92% while the walkers held steady: trackers lose ground as position error builds.

7On small commands, two robust policies barely move

gr00t_wbc_h074_p000 moves the commanded way in 97% of small walking commands and 99% of small turns. htd_wbc and homie score 99–101%: a deadzone. The operator cannot make the last ten centimetres of an approach. supermimic_dyn and morphmimic with the GR00T WBC planner policy come second and third, at 45% and 54%. More in Walk slowly and precisely.

8Stiffer legs are not a free upgrade

Every policy was rerun with the kp and kd of its twelve leg joints scaled by 1.5 and 2.0. At 2×, 43 of the 71 rows get worse; at 1.5×, 27. Most SuperMimic rows survive longer at 1.5×, and every row that completes enough to measure spends more energy and vibrates more.

9The engines agree on the field, not on the pick

Across all 69 ranked rows, survival in the two engines correlates at Spearman 0.97: they agree on who walks and who falls. Among the thirteen leaders, completion correlates at only 0.33. MuJoCo’s best is gr00t_wbc at 0.66 m, 88%; PhysX’s is gr00t_wbc at 0.70 m, 84%. Read both before picking a posture.

10A permissively licensed policy is level with the top

Of the forty-three, two are non-commercial (homie and bfm_zero, CC BY-NC), one is sim-only at its authors’ request, thirteen declare no licence and four grant none for the weights. htd_wbc is BSD-3-Clause: best on position error, cheapest walk in MuJoCo, level on completion, though not on small commands. Terms are in the NOTICE.

11Limitations

• Simulation, not a floor: a shortlist, not a verdict.
• Cost columns cover only the rows that finish often enough.
• Tracker references are ours, never tuned by their authors; a planner-policy row partly measures its planner policy.
• This article measures how publicly released walking policies survive and steer in simulation; it doesn’t explain any stack’s internals.

12Measure your own stack

Everything above runs from the repository on an NVIDIA GPU (the Makefile targets sm_89 by default).

Read completion with an interval. Count the runs whose outcome is complete and put a Wilson 95% interval around the share: 192 of 256 is 75%, or anywhere from 69% to 80%. If two intervals overlap, run more run ids.

Adding a policy. Implement the seven methods of policy_api::Policy — init, step, kp, kd, owned, limits and name — then add the include, a factory branch and its NAMES entry.

13Try it yourself

Twenty-three of them run here: each downloads its real weights and runs at 50 Hz against MuJoCo in your browser, ported from the benchmark’s own policy.cpp. gr00t_wbc runs at 0.66 m, htd_wbc at 0.66 m / 0.12 rad (their best completion rows, not our teleop pick). Most fall over: below the top seven of these twenty-three, only run_residual completes over 1% (6/9%).

Starting on frozen — the stance the crane holds, with no policy running. Pick one above and it downloads that policy’s weights, a few MB on top of the physics engine.

Click it, then drive: W/S forward and back, A/D sideways, Q/E turn. Nothing punches it here, and each policy is capped at its own command range, so amo will not turn.