RL-augmented MPC Framework for Agile and Robust Bipedal Footstep Locomotion Planning and Control Seung Hyeon Bang 1, Carlos Arribalzaga Jov e 1, 2, and Luis Sentis 1 Abstract -- This paper proposes an online bipedal footstep planning strategy that combines model predictive control (MPC) and reinforcement learning (RL) to achieve agile and robust bipedal maneuvers. While MPC-based foot placement controllers have demonstrated their effectiveness in achieving dynamic locomotion, their performance is often limited by the use of simplified models and assumptions. T o address this challenge, we develop a novel foot placement controller that leverages a learned policy to bridge the gap between the use of a simplified model and the more complex full-order robot system. Specifically, our approach employs a unique combination of an ALIP-based MPC foot placement controller for sub-optimal footstep planning and the learned policy for refining footstep adjustments, enabling the resulting footstep policy to capture the robot's whole-body dynamics effectively. We validate the effectiveness of our framework through a series of experiments using the full-body humanoid robot DRACO 3. The results demonstrate significant improvements in dynamic locomotion performance, including better tracking of a wide range of walking speeds, enabling reliable turning and traversing challenging terrains while preserving the robustness and stability of the walking gaits compared to the baseline ALIP-based MPC approach. I. INTRODUCTION Agile and robust bipedal locomotion is essential for humanoid robots to achieve human-level performance. One of the main challenges in achieving this is designing a footstep policy that enables bipeds to constantly adjust their planned footstep positions to maintain balance as well as to achieve more agile and fast maneuvers, even while traversing adverse environments, such as external disturbances or challenging terrains. In this paper, we present an RL-augmented MPC framework designed to generate a footstep policy for agile and robust bipedal locomotion.