Consider scene understanding problems such as predicting where a person is probably reaching, or inferring the pose of 3D objects from depth images, or inferring the probable street crossings of pedestrians at a busy intersection. This paper shows how to solve these problems using Approximate Bayesian Computation. The underlying generative models are built from realistic simulation software, wrapped in a Bayesian error model for the gap between simulation outputs and real data. The simulators are drawn from off-the-shelf computer graphics, video game, and traffic simulation code. The paper introduces two techniques for speeding up inference that can be used separately or in combination. The first is to train neural surrogates of the simulators, using a simple form of domain randomization to make the surrogates more robust to the gap between the simulation and reality. The second is to adaptively discretize the latent variables using a Tree-pyramid approach adapted from computer graphics. This paper also shows performance and accuracy measurements on real-world problems, establishing that it is feasible to solve these problems in real-time.

We present a high-level overview of our research efforts to build an intelligent robot capable of addressing real-world problems. The UJI Humanoid Robot Torso integrates research accomplishments under the common framework of multimodal active perception and exploration for physical interaction and manipulation. Its main components are three subsystems for visuomotor learning, object grasping and sensor integration for physical interaction. We present the integrated architecture and a summary of employed techniques and results. Our contribution to the integrated design of an intelligent robot is in this combination of different sensing, planning and motor systems in a novel framework.