1.5M Steps 3.1M Steps RND BeBold 6.4M Steps 4.6M Steps 7.5M Steps 9.8M Steps 1.0M Steps 1.4M Steps 3.4M Steps 2.4M Steps 3.9M Steps 4.8M Steps

We provide final testing performance for NovelD and all baselines in MiniGrid. We also provide more intrinsic analysis similar to Sec. 4.2 in a seven-room environment in Figure 1. There are other categories of static environment. The initial position of the agent and goal can be random. The position of the agent and goal is randomized.