An Empirical Investigation of Randomized Defenses against Adversarial Attacks

In recent years, Deep Neural Networks (DNNs) have had a drama tic impact on a variety of problems that were long considered very difficult, e. g., image classification and automatic language translation to name just a few. T he accuracy of modern DNNs in classification tasks is remarkable indeed. At the same time, attackers have devised powerful methods to construct specially-craf ted malicious inputs (often referred to as adversarial examples) that can trick DNNs into mis-classifying them. What is worse is that despite the many defense mechanis ms proposed to protect DNNs against adversarial attacks, attackers are of ten able to circumvent these defenses, rendering them useless. This state of affai rs is extremely worrying, especially since machine learning systems get adopted at sc ale. In this paper, we propose a scientific evaluation methodolog y aimed at assessing the quality, efficacy, robustness and efficiency of randomiz ed defenses to protect DNNs against adversarial examples. Using this methodology, we evaluate a variety of defense mechanisms. In addition, we also propose a d efense mechanism we call Randomly Perturbed Ensemble Neural Networks (RPENN s). W e provide a thorough and comprehensive evaluation of the considered d efense mechanisms against a white-box attacker model, six different adversar ial attack methods and using the ILSVRC2012 validation data set.