Learning Velocity-based Humanoid Locomotion: Massively Parallel Learning with Brax and MJX

Recent interest in humanoid robots as general purpose robots has lead to a significant increase in humanoid robotics research and development in both industry and academia. A main reason for the interest is because humanoid robots have the ability to perform repetitive and dull tasks in human environments. A core skill necessary for many tasks, like moving boxes around a warehouse, is robust locomotion. Locomotion planning and control algorithms vary greatly from linear inverted pendulum walking (LIPM) [4] to online whole-body MPC walking [3]. Reinforcement learning (RL) has also been a method of choice recently for robotic motion generation given its ability to adapt to different environments or conditions and generalize well to many scenarios.