A Closer Look at the Calibration of Differentially Private Learners

Modern deep learning models tend to memorize their training data in order to generalize better [1, 2], posing great privacy challenges in the form of training data leakage or membership inference attacks [3, 4, 5]. To address these concerns, differential privacy (DP) has become a popular paradigm for providing rigorous privacy guarantees when performing data analysis and statistical modeling based on private data. In practice, a commonly used DP algorithm to train machine learning (ML) models is DP-SGD [6]. The algorithm involves clipping per-example gradients and injecting noises into parameter updates during the optimization process. Despite that DP-SGD can give strong privacy guarantees, prior works have identified that this privacy comes at a cost of other aspects of trustworthy ML, such as degrading accuracy and causing disparate impact [2, 7, 8]. These tradeoffs pose a challenge for privacy-preserving ML, as it forces practitioners to make difficult decisions on how to weigh privacy against other key aspects of trustworthiness. In this work, we expand the study of privacy-related tradeoffs by characterizing and proposing mitigations for the privacy-calibration tradeoff. The tradeoff is significant as accessing model uncertainty is important for deploying models in safety-critical scenarios like healthcare and law where explainability [9] and risk control [10] are needed in addition to privacy [11]. 1