Invariant Transform Experience Replay

Yijiong Lin 1, Jiancong Huang 1, Matthieu Zimmer 2, Juan Rojas 1, Paul Weng 2 Abstract -- Deep reinforcement learning (DRL) is a promising approach for adaptive robot control, but its current application to robotics is currently hindered by high sample requirements. We propose two novel data augmentation techniques for DRL based on invariant transformations of trajectories in order to reuse more efficiently observed interaction. The first one called Kaleidoscope Experience Replay exploits reflectional symmetries, while the second called Goal-augmented Experience Replay takes advantage of lax goal definitions. In the Fetch tasks from OpenAI Gym, our experimental results show a large increase in learning speed. I. INTRODUCTION Deep reinforcement learning (DRL) has demonstrated great promise in recent years [1], [2]. However, despite being shown to be a viable approach in robotics [3], [4], DRL still suffers from low sample efficiency in practice--an acute issue in robot learning. Given how critical this issue is, many diverse propositions have been presented. For brevity, we only recall the most related to our work.