The study of private inference has been sparked by growing concern regarding the analysis of data when it stems from sensitive sources. We present the first method for private Bayesian inference in exponential families that properly accounts for noise introduced by the privacy mechanism. It is efficient because it works only with sufficient statistics and not individual data. Unlike other methods, it gives properly calibrated posterior beliefs in the non-asymptotic data regime.

This paper proposes an approach for differentially private estimation of the posterior distribution in conjugate exponential-family models. Similar to previous "naive" approaches, it enforces privacy by adding Laplace-distributed noise to the sufficient statistic. Where a naive approach would treat this noisy statistic as true, the main contribution of this paper is a Gibbs sampling algorithm to integrate over uncertainty in the true statistic given the observed noisy statistic. This is the proper Bayesian procedure, and the experiments show that this yields better-calibrated posterior estimates than naive updating or one-posterior sampling (OPS). The paper is very clear, cleanly written and easy to follow; I found no obvious mistakes.

The study of private inference has been sparked by growing concern regarding the analysis of data when it stems from sensitive sources. We present the first method for private Bayesian inference in exponential families that properly accounts for noise introduced by the privacy mechanism. It is efficient because it works only with sufficient statistics and not individual data. Unlike other methods, it gives properly calibrated posterior beliefs in the non-asymptotic data regime. Papers published at the Neural Information Processing Systems Conference.