Differential privacy (DP) Dwork et al. (2006b), Dwork et al. (2006a) is a strong standard to guarantee the privacy for algorithms that have been widely applied to modern machine learning (Abadi et al., 2016). It characterizes the privacy loss via statistical hypothesis testing, thus allowing the mathematically rigorous analysis of the privacy bounds. When multiple operations on the data are involvedand each intermediate step is revealed, composition theorems can be used to keep track of the privacy loss, which combines subadditively (Kairouz et al., 2015). However, because such results are required to be general, their associated privacy bounds are inevitably loose. In contrast, privacy amplification provides a privacy budget for a composition of mechanisms that is less that the budget of each individual operation, which strengthens the bound the more operations are concatenated. Classic examples of this feature are privacy amplification by subsampling (Chaudhuri and Mishra, 2006; Balle et al., 2018), by shuffling (Erlingsson et al., 2019) and by iteration (Feldman et al., 2018; Asoodeh et al., 2020). In this paper, we focus on the setting of privacy amplification by iteration, and extend the analysis via contraction coefficient proposed by Asoodeh et al. (2020) to prove results that apply to an algorithm commonly used in practice, in which the entire dataset is 1

We investigate the framework of privacy amplification by iteration, recently proposed by Feldman et al., from an information-theoretic lens. We demonstrate that differential privacy guarantees of iterative mappings can be determined by a direct application of contraction coefficients derived from strong data processing inequalities for f -divergences. In particular, by generalizing the Dobrushin's contraction coefficient for total variation distance to an f -divergence known as E Differential privacy (DP) [1, 2] has become the standard definition for designing privacy-preserving machine learning algorithms. One reason for its success is its operational significance, which can be best described in terms of binary hypothesis testing (see, e.g., [3, 4]). Nevertheless, it is often difficult to compute DP guarantees for applications where a high number of data accesses is needed for a single analysis [5, 6]. To obtain the DP parameters in such applications, which include machine learning models trained using stochastic gradient descent (SGD), one needs to resort to composition theorems which are often loose due to their generality. As a remedy, several variants of DP have been recently proposed [7-10] based on Rényi divergence. These variants enjoy better composition properties.