A Proofs

A.1 Proof of Theorem 3.1 First we set up some notation. All algorithms we are considering, if not discrete, induce a density w.r.t. the Lebesgue measure. The only difference between Theorem 3.1 and this theorem is that a privacy filter halts at a random The same argument can be used to bound the other direction of the divergence. Since we run batch gradient descent and not SGD as in the library example, we tune all hyperparameters from scratch. We think of the minimum of an empty set as .