Adversarial Examples: Opportunities and Challenges

Abstract--With the advent of the era of artificial intelligence (AI), deep neural networks (DNNs) have shown huge superiority over human in image recognition, speech processing, autonomous vehicles and medical diagnosis. However, recent studies indicate that DNNs are vulnerable to adversarial examples (AEs) which are designed by attackers to fool deep learning models. Different from real examples, AEs can hardly be distinguished from human eyes, but mislead the model to predict incorrect outputs and therefore threaten security critical deep-learning applications. In recent years, the generation and defense of AEs have become a research hotspot in the field of AI security. This article reviews the latest research progress of AEs. First, we introduce the concept, cause, characteristic and evaluation metrics of AEs, then give a survey on the state-of-the-art AE generation methods with the discussion of advantages and disadvantages. After that we review the existing defenses and discuss their limitations. Finally, the future research opportunities and challenges of AEs are prospected. In the era of AI, DNNs have shown great advantages in autonomous vehicles, robotics, network security, image/speech recognition and natural language processing (NLP). For example, in 2017, an intelligent robot with the superior face recognition ability, named XiaoDu developed by Baidu, defeated a representative from the team of humans strongest brain with the score of 3:2 [1]. On October 19th, 2017, the DeepMind team of Google released the AlphaGo Zero, which shocked the world. Compared with the previous AlphaGo, AlphaGo Zero relies on reinforcement learning without any priori knowledge to grow chess skills and finally beats every human competitor [2]. For AI research, the United States received huge support from the government, such as the Federal Research Fund. In October 2016, the United States issued the project of Preparing for the Future of Artificial Intelligence and the National Artificial Intelligence Research and Development Strategic Plan, which raised AI to the national strategic level and formulated ambitious blueprints [3], [4]. Manuscript received xxx; revised xx; accepted xxx. This work is supported by the National Natural Science Foundation of China (Grant NOs. J. Zhang and X. Jiang are with the College of Computer Science and Electronic Engineering, Hunan University, Changsha 410082, China (email: zhangjiliang@hnu.edu.cn). In the same year, AI was written into the nineteenth National Congress report, which pushed the development of AI industries to a new height and filled the gap in the top-level strategy of AI development [5].