Deep Episodic Memory: Encoding, Recalling, and Predicting Episodic Experiences for Robot Action Execution

Humans are ingenious: We have unique abilities to predict the consequences of observed actions, remember the most relevant experiences from the past, and transfer knowledge from previous observations in order to adapt to novel situations. The episodic memory which encodes contextual, spatial and temporal experiences during development plays a vital role to introduce such cognitive abilities in humans. A core challenge in cognitive robotics is compact and generalizable mechanism which allow for encoding, storing and retrieving spatiotemporal patterns of visual observations. Such mechanisms would enable robots to build a memory system, allowing them to efficiently store gained knowledge from past experiences and both recalling and applying such knowledge in new situations. Inspired by infants that learn by observing and memorizing what adults do in the same visual setting, we investigate in this paper how to extend cognitive abilities of robots to autonomously infer the most probable behavior and ultimately adapt it to the current scene. Considering the situation of the humanoid robot ARMAR-IIIa standing in front of a table with a juice carton (see Figure 1) one can ask what the most suitable action is and how it would best be performed. To achieve this goal, we introduce a novel deep neural network architecture for encoding, storing, and recalling past action experiences in an episodic memory-like manner.