A Detailed Proofs

Consider the variance of the gradient with prioritized sampling. X. (8) 2 where X is defined as before. However, we found common implementations to have inefficient aspects, mainly unnecessary for-loops. Time increase of SAC is provided to give a better understanding of the significance. The OpenAI baselines implementation uses an additional sum-tree to compute the minimum over the entire replay buffer to compute the importance sampling weights.