Reviews: Visual Reinforcement Learning with Imagined Goals

This paper proposes an algorithm for learning goal-conditioned RL policy, in which a goal is defined as a single image. The authors propose to encode a state (an image) to a vector in latent space using variational autoencoder, and define reward functions inside the latent space. The paper shows that such reward function outperforms baseline such as pixel based reward functions. The authors then proposed latent goal relabeling, which generates new goals and rewards given an exist tuple (s, a, s'). Finally, the authors propose goal imagination, which samples goals from latent space during training, essentially allowing training without specifying a particular goal.