AGIL: Learning Attention from Human for Visuomotor Tasks

In end-to-end learning of visuomotor behaviors, algorithms such as imitation learning, reinforcement learning (RL), or a combination of both, have achieved remarkable successes in video games [27], board games [36, 37], and robot manipulation tasks [23, 29]. One major issue of using RL alone is its sample efficiency, hence in practice human demonstration can be used to speedup learning [36, 5, 14]. Imitation learning, or learning from demonstration, follows a student-teacher paradigm, in which a learning agent learns from the demonstration of human teachers [1]. A popular approach is behavior cloning, i.e., training an agent to predict (imitate) demonstrated behaviors with supervised learning methods. Imitation learning research mainly focuses on the student-advancing our understanding of the learning agent-while very little effort is made to understand the human teacher. In this work, we argue that understanding and modeling the human teacher is also an important research issue in this paradigm. Specifically, in visuomotor learning tasks, a key component of human intelligence-the visual attention mechanism-encodes a wealth of information that can be exploited by a learning algorithm. Modeling human visual attention and guiding the learning agent with a learned attention model could lead to significant improvement in task performance. We propose the Attention Guided Imitation Learning (AGIL) framework, in which a learning agent first learns a visual attention model from human gaze data, then learns how to perform the visuomotor task from human decision data.