Recent workhasindicated that,unlikehumans, ImageNet-trained CNNs tendto classify images by texture rather than by shape. How pervasiveis this bias, and wheredoesitcomefrom? Wefindthat,whentrainedondatasets ofimageswith conflicting shape and texture, CNNs learn to classify by shape at least as easily as by texture. What factors, then, produce the texture bias in CNNs trained on ImageNet?

Recent work has indicated that, unlike humans, ImageNet-trained CNNs tend to classify images by texture rather than by shape. How pervasive is this bias, and where does it come from? We find that, when trained on datasets of images with conflicting shape and texture, CNNs learn to classify by shape at least as easily as by texture. What factors, then, produce the texture bias in CNNs trained on ImageNet? Different unsupervised training objectives and different architectures have small but significant and largely independent effects on the level of texture bias. However, all objectives and architectures still lead to models that make texture-based classification decisions a majority of the time, even if shape information is decodable from their hidden representations. The effect of data augmentation is much larger. By taking less aggressive random crops at training time and applying simple, naturalistic augmentation (color distortion, noise, and blur), we train models that classify ambiguous images by shape a majority of the time, and outperform baselines on out-of-distribution test sets. Our results indicate that apparent differences in the way humans and ImageNet-trained CNNs process images may arise not primarily from differences in their internal workings, but from differences in the data that they see.

Higher learning rates produce greater shape bias. As shown in Figure A.1, higher values of learning rate and weight decay were associated with greater We found that random-crop augmentation biases models towards texture (Section 5). We did not change the aspect ratio or other data augmentation settings. Networks with limited receptive fields learn texture more easily than shape. Shape is persistently more decodable through the convolutional layers of AlexNet than is texture, which rises through them.

The double descent phenomenon, which deviates from the traditional bias-variance trade-off theory, attracts considerable research attention; however, the mechanism of its occurrence is not fully understood. On the other hand, in the study of convolutional neural networks (CNNs) for image recognition, methods are proposed to quantify the bias on shape features versus texture features in images, determining which features the CNN focuses on more. In this work, we hypothesize that there is a relationship between the shape/texture bias in the learning process of CNNs and epoch-wise double descent, and we conduct verification. As a result, we discover double descent/ascent of shape/texture bias synchronized with double descent of test error under conditions where epoch-wise double descent is observed. Quantitative evaluations confirm this correlation between the test errors and the bias values from the initial decrease to the full increase in test error. Interestingly, double descent/ascent of shape/texture bias is observed in some cases even in conditions without label noise, where double descent is thought not to occur. These experimental results are considered to contribute to the understanding of the mechanisms behind the double descent phenomenon and the learning process of CNNs in image recognition.

The paper is very well written, seemingly involves a massive amount of wordload, and answers most of the questions clearly with evidence and offer conjectures of the unanswerable questions to guide future research. Despite the high quality, I noticed several drawbacks and suggest the authors to address them. In the abstract, the paper says the differences "arise not from differences in their internal workings, but from differences in the data that they see", which seems to suggest that whether the model learns texture or shape primarily depends on the data seen, yet in the experiments, the authors demonstrate that, with more carefully designed regularizations (termed as "self-supervised losses" in the paper), the model can be pushed to focus more on the shape. This empirical observation seems to contradict with the main claim in the abstract since I suppose losses are one of the "internal workings" (or what does "internal workings" mean exactly?). I suggest the authors to revise corresponding texts to reflect this more accurately.

This paper studies an important question and the reviewers found the results convincing enough about the main claim of data augmentation mechanisms contributing to texture bias.

Recent work has indicated that, unlike humans, ImageNet-trained CNNs tend to classify images by texture rather than by shape. How pervasive is this bias, and where does it come from? We find that, when trained on datasets of images with conflicting shape and texture, CNNs learn to classify by shape at least as easily as by texture. What factors, then, produce the texture bias in CNNs trained on ImageNet? Different unsupervised training objectives and different architectures have small but significant and largely independent effects on the level of texture bias.