Constraining Gaussian Process Implicit Surfaces for Robot Manipulation via Dataset Refinement

--Model-based control faces fundamental challenges in partially-observable environments due to unmodeled obstacles. We propose an online learning and optimization method to identify and avoid unobserved obstacles online. Our method, Constraint Obeying Gaussian Implicit Surfaces (COGIS), infers contact data using a combination of visual input and state tracking, informed by predictions from a nominal dynamics model. We then fit a Gaussian process implicit surface (GPIS) to these data and refine the dataset through a novel method of enforcing constraints on the estimated surface. This allows us to design a Model Predictive Control (MPC) method that leverages the obstacle estimate to complete multiple manipulation tasks. By modeling the environment instead of attempting to directly adapt the dynamics, our method succeeds at both low-dimensional peg-in-hole tasks and high-dimensional deformable object manipulation tasks. Our method succeeds in 10/10 trials vs 1/10 for a baseline on a real-world cable manipulation task under partial observability of the environment. PECIAL care must be taken when using model-based planning and control methods in partially observable environments. This is particularly important where not all obstacles are modeled by dynamics, to avoid collisions with unmodeled or unobserved parts of the environment. Such collisions could prevent task completion; for instance, the object being manipulated might be blocked by the unmodeled environment object. The challenge is heightened when manipulating deformable objects like cables in the home or office. This creates more possibilities for task failure.