High-frequency component helps explain the generalization of convolutional neural networks

There are many works aiming to explain the generalization behavior of neural networks using heavy mathematical machinery, but we will do something different here: with a simple and intuitive twist of data, we will show that many generalization mysteries (like adversarial vulnerability, BatchNorm's efficacy, and the "generalization paradox") might be results of our overconfidence in processing data through naked eyes. The models may have not outsmarted us, but the data has. Let's start with an interesting observation (Figure 1): we trained a ResNet-18 with the Cifar10 dataset, picked a test sample, and plotted the model's prediction confidence for this sample. Then we mapped the sample into the frequency domain through Fourier transform, and cut the frequency representation into its high-frequency component (HFC) and low-frequency component (LFC). Although this phenomenon can only be observed with a subset of samples ( 600 images), it's striking enough to raise an alarm.