Effect of Ambient-Intrinsic Dimension Gap on Adversarial Vulnerability

The existence of adversarial attacks on machine learning models imperceptible to a human is still quite a mystery from a theoretical perspective. In this work, we introduce two notions of adversarial attacks: natural or onmanifold attacks, which are perceptible by a human/oracle, and unnatural or off-manifold attacks, which are not. We argue that the existence of the off-manifold attacks is a natural consequence of the dimension gap between the intrinsic and ambient dimensions of the data. For 2-layer ReLU networks, we prove that even though the dimension gap does not Figure 1: Mental image: The oracle decision boundary affect generalization performance on samples (green dashed line) determines the label (blue or red) drawn from the observed data space, it makes of any point in the Euclidean space. The observed data the clean-trained model more vulnerable to space consists of 1-dimensional line segments immersed adversarial perturbations in the off-manifold in the 2-dimensional space. The model learns the direction of the data space. Our main results estimated decision boundary (black dotted line) based provide an explicit relationship between the on the observed data.