Reinforcement Learning Helping Humanoids Walk

Humanoids have gained immense commercial traction in the last decade and accelerated research in the field of biped locomotion, perception, and end-to-end task completion. End-to-end robot learning has become the latest norm in robot research that combines sensing, planning, and control into one fused unit that can perform desired tasks without any manual tweaking.

Stable and agile bipedal locomotion is the most critical feature for humanoids to be successful. It allows robots to perform tasks using the dexterity of the robot’s multiple degrees of freedom. This article explains how reinforcement learning was recently used by researchers at UC Berkeley for humanoid locomotion demonstrated on Agility Robotics’ Digit robot.

Bipedal Gait

Classical research in biped gait focussed on statically stable biped gait primarily using the Zero Moment Point criterion. It keeps the robot balanced during every phase of a step since the center of mass stays in the convex hull ( or support polygon). Robots like Honda’s Asimo and most likely Tesla’s Optimus use a ZMP-like controller. Dynamically stable gait models steps based on an inverse pendulum strategy and the biped is not stable during the leg’s swing phase. Dynamic gait allows faster walking than static gait but is a very challenging controller to implement.

Reinforcement Learning

Reinforcement Learning has had its ebbs and flows in the past decade of Artificial…