We thank the reviewers for their close reading of the paper and helpful feedback. Forexample, one can use thedensity ratio estimates7 provided by DualDICE to modify (importance-weight) the off-policy data distribution before passing it to a policy8 gradient orQ-learning method. The figures are overall too small... In Figure 2 the x axis label is missing. The x-axis is training step.

The key challenge is that the existing literature has only focused on designing algorithms with minimalcomputation, ignoring the fact that the practical latency can also be influenced byscheduling strategiesand hardware properties.

As[R1]38 has pointed out, our novel interpretation of KL term gives new insights and variations on online Bayesian learning.39 Since UCL samples the weight parameters only once for each iteration, applying it to actor-critic based42 reinforcement learning algorithm becomes possible.

All hyper-parameters are the same as the ones used in the paper or are default to A2C. The same set of auxiliary tasks are also used. Ability to separate harmful auxiliary tasks: In Figure 3 of the orignal paper, we show that AutoEncoder is a11 harmful auxiliary task for Finger Turn environment. Here, a toy example in Figure 7 with one positive auxiliary12 task and one harmful auxiliary task shows that our algorithm is able to avoid adversarial auxiliary tasks without13 any prior knowledge.

Horovod: fast and easy distributed deep learning in tensorflow.

Notation and conventions: [n] = {1,,n}; |S| is the size of setP; dxe is the smallest integer greater thanx; bxc is the largest integer smaller thanx; kXk is the nuclear norm of matrixL, i.e., the sum of the singular values of matrixX; Z+ is the set of positive integers;Z i is the set of integers which are greater thani; Given S1, S2, the reduction ofS1 by S2 is denoted as S1\S2={i S1:i / S2};finally,A(n) B(n)meansA(n)/B(n) 1asn .

We adopt four bioinformatics datasets in the experiment. Given the input graph, it will randomly add or cut a certain portion ofconnections between nodes withtheprobability of0.2. It will set the feature of 20% nodes in the graph to Gaussian noises with mean and standard deviation is 0.5. We adopt the Adam [5] optimizer, which is a variant of Stochastic Gradient Descent (SGD) with adaptivemoment estimation.

(iteration 2).

This is the state-of-the-art graph contrastive learning based recommendation method, which proposes randomly node dropout, edge dropout, and random walk for augmentation onthebipartite graph.