Data Poisoning Attacks against Online Learning

As machine learning algorithms are increasing used in security-critical applications, there is a growing need to design them with active adversaries in mind. A class of adversarial attacks on machine learning that have received much attention is data poisoning attacks [21, 20, 19, 4, 7, 11]. Here, an adversary is aware of the learner's training data and algorithm, and has the power to alter a small fraction of the training data in order to make the trained classifier satisfy certain objectives. For example, a sabotage adversary may try to degrade the overall accuracy of the trained classifier as part of an industrial sabotage campaign, or a profit-oriented adversary may try to poison the training data so that the resulting model favors it - say, by recommending the its products over others. While there has been a long line of prior work on data poisoning [21, 20, 19, 4, 7, 11, 3, 8], most of it has focussed in the offline setting, where a classifier or some other model is trained on a fixed input.