Perceptive Locomotion through Nonlinear Model Predictive Control

Abstract--Dynamic locomotion in rough terrain requires accurate foot placement, collision avoidance, and planning of the underactuated dynamics of the system. Reliably optimizing for such motions and interactions in the presence of imperfect and often incomplete perceptive information is challenging. We present a complete perception, planning, and control pipeline, that can optimize motions for all degrees of freedom of the robot in real-time. To mitigate the numerical challenges posed by the terrain a sequence of convex inequality constraints is extracted as local approximations of foothold feasibility and embedded into an online model predictive controller. Steppability classification, plane segmentation, and a signed distance field are precomputed per elevation map to minimize the computational effort during the optimization. In the shown configuration, the top foothold is 60 cm above the lowest foothold. These approaches build on a strict hierarchy of first selecting footholds and optimizing torso motion afterward. Still, complex terrains that jointly optimized has shown impressive results in simulation require precise foot placements, e.g., negative obstacles and [18]-[20] and removes the need for engineered torsofoot stepping stones as shown in Figure 1, remain difficult. Complex motions can be automatically A key challenge lies in the fact that both the terrain and discovered by including the entire terrain in the optimization. Additionally, due to the non-convexity, nonlinearity, mature methods exist for perceptive locomotion with a slow, and discontinuity introduced by optimizing over static gait [4]-[8] and for blind, dynamic locomotion that arbitrary terrain, these methods can get stuck in poor local assumes flat terrain [9]-[11]. Dedicated work on providing an initial guess is recently shown the ability to generalize blind locomotion needed to find feasible motions reliably [21]. Still, tightly integrating perception to achieve coordinated and This work presents a planning and control framework precise foot placement remains an active research problem.