Deep Forward and Inverse Perceptual Models for Tracking and Prediction

Several fundamental problems in robotics, including state estimation, prediction, and motion planning rely on accurate models that can map state to measurements (forward models) or measurements to state (inverse models). Classic examples include the measurement models for global positioning systems, inertial measurement units, or beam sensors that are frequently used in simultaneous localization and mapping [1], or the forward and inverse kinematic models that map joint configurations to workspace and vice-versa. Some of these models can be very difficult to derive analytically, and, in these cases, roboticists have often resorted to machine learning to infer accurate models directly from data. For example, complex nonlinear forward kinematics have been modeled with techniques as diverse as Bayesian networks [2] and Bezier Splines [3], and many researchers have tackled the problem of learning inverse kinematics with nonparametric methods like locally weighted projection regression (LWPR) [4], [5], mixtures of experts [6], and Gaussian Process Regression [7]. While these techniques are able to learn accurate models, they rely heavily on prior knowledge about the kinematic relationship between the robot statespace and workspace. Despite the important role that forward and inverse models have played in robotics, there has been little progress in defining these models for very high-dimensional sensor data like images and video.