Active Exploration for Robotic Manipulation

Abstract-- Robotic manipulation stands as a largely unsolved problem despite significant advances in robotics and machine learning in recent years. One of the key challenges in manipulation is the exploration of the dynamics of the environment when there is continuous contact between the objects being manipulated. This paper proposes a model-based active exploration approach that enables efficient learning in sparse-reward robotic manipulation tasks. The proposed method estimates an information gain objective using an ensemble of probabilistic models and deploys model predictive control (MPC) to plan actions online that maximize the expected reward while also performing directed exploration. We evaluate our proposed algorithm in simulation and on a real robot, trained from Figure 1: Our active exploration strategy evaluated on a challenging scratch with our method, on a challenging ball pushing task Tilted Pushing task in simulation (left) and on the real robot (right). Our real-world robot experiment serves as sparse reward model in order to bring the ball to a target location. We believe that for of dexterous manipulation capabilities was one of the major robots to reach human-level manipulation skills, they must driving factors in the development of the human mind [1] actively explore and adapt to new instances of a task. Performing manipulation is cognitively highly demanding, forcing the We define active exploration as the directed search of the agent to reason not only about the impact of its actions on agent, during the learning process, for unvisited state-action itself, but also on the environment. This inherent complexity pairs that would maximize the agent's performance.