Considerations for differentially private learning with large-scale public pretraining – interview with Gautam Kamath

Florian Tramer, Gautam Kamath and Nicholas Carlini won an International Conference on Machine Learning (ICML2024) best paper award for their work Position: Considerations for Differentially Private Learning with Large-Scale Public Pretraining. Differential privacy is a rigorous and provable notion of data privacy. Among other things, training a machine learning model with differential privacy can prevent it from spitting out its training data. The issue is that training a model with differential privacy generally comes at a significant hit to a model's utility. By incorporating "public data" (i.e., data that is not subject to privacy constraints) into the training procedure, it can help alleviate this concern and increase the resulting model's utility.