Exploration in sparse-reward reinforcement learning (RL) is difficult due to the need for long, coordinated sequences of actions in order to achieve any reward.

This challenge necessitates new mathematical and algorithmic tools that enable adaptation and zero-shot transfer.

In fact, without such assumptions, this problem is NP-hard [Sim02; MR18].

Based on our large-scale study, we summarize our key findings as follows: Key findings.