Fundamental Limits of Membership Inference Attacks on Machine Learning Models

In today's data-driven era, machine learning models are designed to reach higher performance, and the size of new models will then inherently increase, therefore the information stored (or memorized) in the parameters [Hartley and Tsaftaris, 2022, Del Grosso et al., 2023]. The protection of sensitive information is of paramount importance. Membership Inference Attacks (MIAs) have emerged as a concerning threat, capable of unveiling whether a specific data point was part of the training dataset of a machine learning model [Shokri et al., 2017, Nasr et al., 2019, Song et al., 2017a, Zhu et al., 2019]. Such attacks can potentially compromise individual privacy and security by exposing sensitive information [Carlini et al., 2023a]. Furthermore, a recent publication [Tabassi et al., 2019] from the National Institute of Standards and Technology (NIST) explicitly notes that an MIA that successfully identifies an individual as part of the dataset used for training the target model constitutes a breach of confidentiality. To date, the most comprehensive defense mechanism against privacy attacks is differential privacy (DP), a framework initially introduced by Dwork et al. [2006]. DP has shown remarkable adaptability in safeguarding the privacy of machine learning models during training, as demonstrated by the works of Hannun et al. [2021], Jayaraman and Evans [2019]. However, it is worth noting that achieving a high level of privacy through differentially private training often comes at a significant cost to the accuracy of the model, especially when aiming for a low privacy parameter [Sablayrolles et al., 2019]. Conversely, when evaluating the practical effectiveness of DP in terms of its ability to protect against privacy attacks empirically, the outlook is considerably more positive.