Metamers of neural networks reveal divergence from human perceptual systems

We generated model metamers for natural stimuli by performing gradient descent on a noise signal, matching the responses of individual layers of image and audio networks to a natural image or speech signal. The resulting signals reflect the invariances instantiated in the network up to the matched layer. We then measured whether model metamers were recognizable to human observers - a necessary condition for the model representations to replicate those of humans.