Does Label Differential Privacy Prevent Label Inference Attacks?

Intuitively, label-DP presents an easier task for the learner compared to DP since the training features are assumed to be public. Indeed, prior work showed that label-DP learning Label differential privacy (label-DP) is a popular algorithms can achieve much higher test accuracy compared framework for training private ML models on to the best DP counterparts on benchmark datasets. However, datasets with public features and sensitive private such models also attain a high accuracy on the training labels. Despite its rigorous privacy guarantee, it set, which enables an adversary to simply evaluate the model has been observed that in practice label-DP does on the public training features to (correctly) predict the private not preclude label inference attacks (LIAs): Models labels (Busa-Fekete et al., 2021)--a method that we trained with label-DP can be evaluated on the refer to as the simple prediction attack (SPA). The existence public training features to recover, with high accuracy, of such a paradoxical adversary raises the question the very private labels that it was designed of whether label-DP is truly a meaningful privacy notion to to protect. In this work, we argue that this phenomenon strive for.