Combining model-predictive control and predictive reinforcement learning for stable quadrupedal robot locomotion

Modern quadrupedal robots are highly praised for their high degree of mobility, maneuverability, and ability to traverse through diverse terrains, making them well-suited for applications such as inspection and delivery [1, 2]. However, these robots' performance comes at the price of complex mechanical structures with many degrees of freedom to control. As a result, developing control systems that enable these robots to efficiently move and operate in dynamically changing environments is one of the most important tasks in the field of quadruped robotics research. Model predictive control (MPC) includes a range of widely adopted control methods [3] not only capable of efficiently handling industrial problems that entail complex processes [4, 5, 6], but that has also been successfully applied to other problems such as indoor microclimate control given its ability to operate over finite prediction horizons [?, 7, 8, 9]. In one of the early works on MPC's application to the motion of a real quadrupedal robot [10], the nonlinear optimization problem was converted into a quadratic program (QP) by linearizing the system dynamics along a desired walking trajectory. Importantly, the accuracy of the immediate robot dynamics approximation turned out to be more significant than that of the robot dynamics approximation over the prediction horizon. Notwithstanding the advantages of linearization of the system dynamics, nonlinear MPC finds application for quadrupedal robots [11]. As traditional MPC solvers can suffer from short planning horizon, convergence to local optima, errors in the dynamics model, and failure to account for future replanning [12]. Several new MPC-based control schemes have been introduced, each aiming to improve specific characteristics of interest.