Dream to Adapt: Meta Reinforcement Learning by Latent Context Imagination and MDP Imagination

Meta reinforcement learning (Meta RL) has been amply explored to quickly learn an unseen task by transferring previously learned knowledge from similar tasks. However, most state-of-the-art algorithms require the meta-training tasks to have a dense coverage on the task distribution and a great amount of data for each of them. In this paper, we propose MetaDreamer, a context-based Meta RL algorithm that requires less real training tasks and data by doing meta-imagination and MDPimagination. We perform meta-imagination by interpolating on the learned latent context space with disentangled properties, as well as MDP-imagination through the generative world model where physical knowledge is added to plain VAE networks. Our experiments with various benchmarks show that MetaDreamer outperforms existing approaches in data efficiency and interpolated generalization. Meta reinforcement learning has been widely explored to enable quick adaptation to new tasks by learning "how to learn" across a set of meta-training tasks. State-of-the-art meta-learning algorithms have achieved great success in adaptation efficiency and generalization. However, these algorithms rely heavily on abundant data of a large number of diverse meta-training tasks and dense coverage on the task distribution, which may not always be available in real-world.