Review -- What Makes Instance Discrimination Good for Transfer Learning?

The contrastive network provides more complete reconstructions spatially. The images are reconstructed at the correct scale and location. A possible explanation is that in order to make one instance unique from all other instances, the network strives to preserve as much information as possible. The supervised network loses information over large regions in the images, likely because its features are mainly attuned to the most discriminative object parts, which are central to the classification task, rather than objects and images as a whole. The resulting loss of information may prevent the supervised network from detecting the full envelope of the object.