Understanding Visual Feature Reliance through the Lens of Complexity

Recent studies suggest that deep learning models' inductive bias towards favoring simpler features may be an origin of shortcut learning. Yet, there has been limited focus on understanding the complexities of the myriad features that models learn. In this work, we introduce a new metric for quantifying feature complexity, based on V-information and capturing whether a feature requires complex computational transformations to be extracted. Using this V-information metric, we analyze the complexities of 10,000 features--represented as directions in the penultimate layer--that were extracted from a standard ImageNet-trained vision model. Our study addresses four key questions:First, we ask what features look like as a function of complexity, and find a spectrum of simple-to-complex features present within the model.