pnsgd
- Research Report > New Finding (1.00)
- Research Report > Experimental Study (1.00)
- Research Report > New Finding (1.00)
- Research Report > Experimental Study (1.00)
- Information Technology > Security & Privacy (1.00)
- Law (0.67)
- Information Technology > Security & Privacy (1.00)
- Information Technology > Information Management (0.92)
- Information Technology > Artificial Intelligence > Representation & Reasoning (0.68)
- Information Technology > Artificial Intelligence > Machine Learning > Statistical Learning > Gradient Descent (0.47)
Privacy Amplification via Iteration for Shuffled and Online PNSGD
Sordello, Matteo, Bu, Zhiqi, Dong, Jinshuo
Differential privacy (DP) Dwork et al. (2006b), Dwork et al. (2006a) is a strong standard to guarantee the privacy for algorithms that have been widely applied to modern machine learning (Abadi et al., 2016). It characterizes the privacy loss via statistical hypothesis testing, thus allowing the mathematically rigorous analysis of the privacy bounds. When multiple operations on the data are involvedand each intermediate step is revealed, composition theorems can be used to keep track of the privacy loss, which combines subadditively (Kairouz et al., 2015). However, because such results are required to be general, their associated privacy bounds are inevitably loose. In contrast, privacy amplification provides a privacy budget for a composition of mechanisms that is less that the budget of each individual operation, which strengthens the bound the more operations are concatenated. Classic examples of this feature are privacy amplification by subsampling (Chaudhuri and Mishra, 2006; Balle et al., 2018), by shuffling (Erlingsson et al., 2019) and by iteration (Feldman et al., 2018; Asoodeh et al., 2020). In this paper, we focus on the setting of privacy amplification by iteration, and extend the analysis via contraction coefficient proposed by Asoodeh et al. (2020) to prove results that apply to an algorithm commonly used in practice, in which the entire dataset is 1