SE3-Nets: Learning Rigid Body Motion using Deep Neural Networks

The ability to predict how an environment changes based on forces applied to it is fundamental for a robot to achieve specific goals. For instance, in order to arrange objects on a table into a desired configuration, a robot has to be able to reason about where and how to push individual objects, which requires some understanding of physical quantities such as object boundaries, mass, surface friction, and their relationship to forces. A standard approach in robot control is to use a physical model of the environment and perform optimal control to find a policy that leads to the goal state. For instance, extensive work utilizing the MuJoCo physics engine [1] has shown how strong physics models can enable solutions to control problems in complex and contact-rich environments [2]. A shortcoming of such models is, however, that they rely on very accurate estimates of the state of the system [3].