"Real Attackers Don't Compute Gradients": Bridging the Gap Between Adversarial ML Research and Practice

According to the few recorded accounts Next we turn our attention to the research domain and of security failures "in the wild," ML systems can be broken take a snapshot of the current landscape of adversarial ML by naïve attackers that are not systematically exploiting the as portrayed in scientific papers ( IV). After surveying the vulnerabilities of ML, but rather are developing attacks by proceedings of the "Top-4" security conferences from 2019 guessing--either indiscriminately or by some coarse heuristic to 2021, we systematically analyze all 88 papers that consider [6], [7]. Red-team exercises on ML systems often take attacks against ML or corresponding defenses. Of these papers, advantage of security gaps that are agnostic to the existence 89% only evaluate algorithms based on neural networks, 63% of an ML model, and subsequent defensive recommendations focus on computer vision, and 80% perform their experiments are likewise more broad than, e.g., adversarial training [8], on "benchmarks". We discover several inconsistencies in the [9]. Additionally, the ML models deployed in productiongrade terminology adopted in reputable prior work. We also identify ML systems are often not directly observable (and are several positive trends, such as an increasing amount of papers sometimes even unreachable) by most attackers [10].