We thank the reviewers for thoroughly commenting on our article; their comments give us the opportunity to improve

For Montezuma's Revenge, the average prediction error is In this case, the irrelevant intrinsic reward completely obscures the target goal. The less information is available about this step, the more uncertain the model and the higher the error. R4, in general we cannot guarantee that the prediction error is a measure of uncertainty. For an intuition about W-MSE representation and stochasticity, let's consider the noisy TV experiment: there is a TV in Atari and compare it with the best-performing methods such as NGU. To show how the seed affects the performance we included Figure 1 with training dynamics in the supplementary.