Sparse and Transferable Universal Singular Vectors Attack

In recent years, deep learning approaches have become increasingly popular in many areas and applications, starting from computer vision Dosovitskiy et al. [2021a] and natural language processing Touvron et al. [2023], Chung et al. [2022] to robotics Roy et al. [2021] and speech recognition Baevski et al. [2020]. The success and availability of pre-trained neural networks have also made it easier for researchers and developers to use these models for their applications. Despite tremendous advances, it was discovered that deep learning models are vulnerable to small perturbations of input data called adversarial attacks that mislead models and cause incorrect predictions Szegedy et al. [2014], Goodfellow et al. [2014], Moosavi-Dezfooli et al. [2017]. Adversarial attacks as a phenomenon first appeared in the field of computer vision and have raised concerns about the reliability in safety-critical machine learning applications. Initially, adversarial examples were constructed for each individual input Szegedy et al. [2014], making it challenging to scale attacking methods to large datasets. In Moosavi-Dezfooli et al. [2017], the authors show the existence of universal adversarial perturbations (UAPs) that result in the model's misclassification for most of the inputs. Such attacks are crucial for adversarial machine learning research, as they are easier to deploy in real-world applications and raise questions about the safety and robustness of state-of-the-art architectures. However, the proposed optimization algorithm requires vast data, making it complicated to fool real-world systems. In contrast, Khrulkov and Oseledets [2018] proposes a sample-efficient method to construct perturbation using leading (p, q)-singular vectors Boyd [1974] These authors contributed equally to this work.