Spurious Features Everywhere -- Large-Scale Detection of Harmful Spurious Features in ImageNet

Spurious Features in Training Data bird feeder graffiti eucalyptus label Benchmark performance of deep learning classifiers alone is not a reliable predictor for the performance of a deployed model. In particular, if the image classifier has picked up spurious features in the training data, its predictions can fail in unexpected ways. In this paper, we develop Hummingbird Freight Car Koala Hard Disc a framework that allows us to systematically identify Images from the web with spurious feature spurious features in large datasets like ImageNet. It is but no class features classified as class below based on our neural PCA components and their visualization. Previous work on spurious features often operates in toy settings or requires costly pixel-wise annotations. In contrast, we work with ImageNet and validate our results by showing that presence of the harmful spurious feature of a class alone is sufficient to trigger the prediction of that class. We introduce the novel dataset "Spurious ImageNet" which allows to measure the reliance of any ImageNet classifier on harmful spurious features. Moreover, we introduce SpuFix as a simple mitigation method to reduce the dependence of any ImageNet classifier on previously identified Hummingbird Freight Car Koala Hard Disc harmful spurious features without requiring additional labels Figure 1: Top: Examples of spurious features found via or retraining of the model. We provide code and data our neural PCA components but not in previous study [61].