Learning Action-Transferable Policy with Action Embedding

Despite achieving great success on performance in various sequential decision task, deep reinforcement learning is extremely data inefficient. Many approaches have been proposed to improve the data efficiency, e.g. Previous researches on transfer learning mostly attempt to learn a common feature space of states across related tasks to exploit knowledge as much as possible. However, semantic information of actions may be shared as well, even between tasks with different action space size. In this work, we first propose a method to learn action embedding for discrete actions in RL from generated trajectories without any prior knowledge, and then leverage it to transfer policy across tasks with different state space and/or discrete action space. Our experimental results show that our method can effectively learn informative action embeddings and accelerate learning by policy transfer across tasks. Introduction Deep reinforcement learning (DRL), which combines reinforcement learning algorithms and deep neural networks, has achieved great success in many domains, such as playing Atari games (Mnih et al. 2015), playing game of Go (Silver et al. 2016) and robotics control (Levine et al. 2016). Although the DRL is viewed as one of the most potential ways to the General Artificial Intelligence, it is still criticized for its data inefficiency. Training an agent from scratch requires considerable numbers of interactions with the environment for a very specific task.