Provably effective detection of effective data poisoning attacks

Dataset poisoning attacks present a threat against machine learning models because they introduce subtle, ostensibly undetectable changes to the data on which the model will be trained. Moreover, attackers often craft attacks to deterministically change a model's behavior by invoking a latent trigger that they set in the resultant model. We will introduce the precise threat model in which we are interested in Section 2. Researchers often frame dataset poisoning and its analysis from the point of view of optimization theory [1]-[5]. E.g., in the computer vision setting, one might attempt to alter as few pixels as possible in as few images as possible while still producing targeted misclassifications. For text generation, one might aim to change as few tokens as possible to as few corpus sentences as possible while causing targeted semantic misalignment on the next phrase or sentence. In general, this vantage is convenient for conducting attacks. Even locally optimizing the criteria for an attack typically yields a dataset that effectively attacks models trained on it. From this perspective, it is natural to also frame detection of data poisoning as an optimization problem. For example, in [4], it is hypothesized that poisoning a dataset impacts the most dominant features in neural networks trained on it and it is shown that under this assumption poisoning can be provably detected by solving an optimization problem.