Deep neural networks (DNNs) are proved to be vulnerable against backdoor attacks. A backdoor is often embedded in the target DNNs through injecting a backdoor trigger into training examples, which can cause the target DNNs misclassify an input attached with the backdoor trigger. Existing backdoor detection methods often require the access to the original poisoned training data, the parameters of the target DNNs, or the predictive confidence for each given input, which are impractical in many real-world applications, e.g., on-device deployed DNNs. We address the black-box hard-label backdoor detection problem where the DNN is fully black-box and only its final output label is accessible. We approach this problem from the optimization perspective and show that the objective of backdoor detection is bounded by an adversarial objective. Further theoretical and empirical studies reveal that this adversarial objective leads to a solution with highly skewed distribution; a singularity is often observed in the adversarial map of a backdoorinfected example, which we call the adversarial singularity phenomenon. Based on this observation, we propose the adversarial extreme value analysis (AEVA) to detect backdoors in black-box neural networks. AEVA is based on an extreme value analysis of the adversarial map, computed from the monte-carlo gradient estimation. Evidenced by extensive experiments across multiple popular tasks and backdoor attacks, our approach is shown effective in detecting backdoor attacks under the black-box hard-label scenarios. Deep Neural Networks (DNNs) have pervasively been used in a wide range of applications such as facial recognition (Masi et al., 2018), object detection (Szegedy et al., 2013), autonomous driving (Okuyama et al., 2018), and home assistants (Singh et al., 2020). In the meanwhile, DNNs become increasingly complex. Training state-of-the-art models requires enormous data and expensive computation.

Getting your driving licence is a milestone moment for many people. You go through rigorous theory and practical tests, sometimes more than once, before you are given the privilege of being on the road. This, of course, is to ensure the safety of the driver, any passengers and other road users, writes Raina Victor of Birketts LLP. You are also aware of the consequences of driving going wrong, including that the liability for any accident falls (for the most part) on the driver. But what about car accidents that are not caused by the driver of the vehicle but the vehicle itself?

It's lunchtime, and the worker bees of Mountain View who aren't interested in their company's own catering are walking down East Middlefield Road in search of grub. It's a lovely esplanade, but all the trees and light poles make these pedestrians hard to spot. That makes things tricky for a human driver, and extra troublesome for a robot trying to learn to work the wheel. But on a large monitor inside Soroush Salehian and Mina Rezk's Mercedes Sprinter van, every meandering biped stands out against a sea of white. Those walking toward the van are blue, those moving away from it are red.