Yet, releasing R(x) might reveal sensitive information on x. Instead, the The Gaussian Mechanism (GM), which consists curator may use a private algorithm A to release a in adding Gaussian noise to a vectorvalued sanitized approximation A(R)(x) of R(x). To guarantee query before releasing it, is a standard that the amount of information leaked by releasing privacy protection mechanism. In particular, A(R)(x) is limited, DP ensures that the distributions given that the query respects some of A(R)(x) and A(R)(y) are close for any y x, i.e., L2 sensitivity property (the L2 distance between that is close to x according to a neighboring relation outputs on any two neighboring inputs (databases that only differ in one row for instance). Several is bounded), GM guarantees Rényi Differential divergences have been considered to measure the Privacy (RDP). Unfortunately, precisely closeness between these two distributions, leading to different bounding the L2 sensitivity can be hard, thus variants of DP. Among them, Rényi-Differential leading to loose privacy bounds. In this work, Privacy (RDP), which is based on the Rényi divergence, we consider a Relative L2 sensitivity assumption, has become popular for its mathematical properties in which the bound on the distance between [Mironov, 2017].

We provide an improved analysis of standard differentially private gradient descent for linear regression under the squared error loss. Under modest assumptions on the input, we characterize the distribution of the iterate at each time step. Our analysis leads to new results on the algorithm's accuracy: for a proper fixed choice of hyperparameters, the sample complexity depends only linearly on the dimension of the data. This matches the dimension-dependence of the (non-private) ordinary least squares estimator as well as that of recent private algorithms that rely on sophisticated adaptive gradient-clipping schemes (Varshney et al., 2022; Liu et al., 2023). Our analysis of the iterates' distribution also allows us to construct confidence intervals for the empirical optimizer which adapt automatically to the variance of the algorithm on a particular data set. We validate our theorems through experiments on synthetic data.