On the Vulnerability of Data Points under Multiple Membership Inference Attacks and Target Models

Abstract--Membership Inference Attacks (MIAs) infer whether a data point is in the training data of a machine learning model. It is a threat while being in the training data is private information of a data point. Intuitively, data points that MIA accurately detects are vulnerable. Considering those data points may exist in different target models susceptible to multiple MIAs, the vulnerability of data points under multiple MIAs and target models is worth exploring. This paper defines new metrics that can reflect the actual situation of data points' vulnerability and capture vulnerable data points under multiple MIAs and target models. From the analysis, MIA has an inference tendency to some data points despite a low overall inference performance. Additionally, we implement 54 MIAs, whose average attack accuracy ranges from 0.5 to 0.9, to support our analysis with our scalable and flexible platform, Membership Inference Attacks Platform (VMIAP). Furthermore, previous methods are unsuitable for finding vulnerable data points under multiple MIAs and different target models. Finally, we observe that the vulnerability is not characteristic of the data point but related to the MIA and target model. Notably, overfitting is the most frequently applications, security and privacy problems are exposed, mentioned reason. Yeom et al. showed that overfitting is a including models' fairness [1], [2], adversarial examples [3], sufficient but unnecessary condition for MIA [10]. A membership inference Long et al. showed a pragmatic MIA to well-generalized attack (MIA) detects whether a data point is in the training models, which also means overfitting is not the necessary data of a machine learning model by which it violates the reason [11]. Yaghini et al. [9], and Da et al. [12] found data points' privacy. MIA became an important topic after that a subgroup of data points with one or several sharing the seminal work by Shokri et al. [7]. A successful MIA has attributes is more vulnerable to MIA.