The running time and memory needed for our algorithm to approximate the privacy curveofaDPalgorithm composed with itselfktimes is O( k).

While this is not close to the state-of-the-art in general (c.f. Figure 2 shows the effect of action entropy on exploratory behavior in Montezuma's Revenge. Number of unique avatar positions visited. Full training curves across all 6 Atari games are shown in Figure 1, including the random policy baseline. To ensure this didn't hamper performance, we At each state visited by the agent evaluator during training, the agent's state (consisting of the avatar's The full curves are included for completeness. The compute cluster we performed experiments on is heterogenous, and has features such as host-sharing, adaptive load-balancing, etc.

Failure case 2: Augerino depends on the used parameterisation of invariance. The full GGN approximation in Eq. 5 is inO(NP2C) for computingN matrix-products. The diagonalGGNapproximation would be inO(NPC)and computation of the log-determinant onlyO(P). Computing the log-determinant can be done efficiently inO(D3 +G3)by decomposing the Kronecker factors (Immer et al., 2021a). The last two terms dependent onS come up due to the aggregation ofaugmentation samples inour approximation, that is,the expectations overaandg in the second line of Eq. 15.