All hyper-parameters are the same as the ones used in the paper or are default to A2C. The same set of auxiliary tasks are also used. Ability to separate harmful auxiliary tasks: In Figure 3 of the orignal paper, we show that AutoEncoder is a11 harmful auxiliary task for Finger Turn environment. Here, a toy example in Figure 7 with one positive auxiliary12 task and one harmful auxiliary task shows that our algorithm is able to avoid adversarial auxiliary tasks without13 any prior knowledge.

We thank the reviewers for the positive feedback on our motivation and algorithm. The updated figure of the submitted paper's Figure 2 is shown on the right. We show in Figure 3 that cosine similarity performs worse when gradient balancing is removed. All hyper-parameters are the same as the ones used in the paper or are default to A2C. The same set of auxiliary tasks are also used.