Unsupervised deep learning identifies semantic disentanglement in single inferotemporal face patch neurons

To answer these questions in the context of face perception, we joined forces with our collaborators at Caltech (Doris Tsao) and the Chinese Academy of Science (Le Chang). We chose faces because they are well studied in the neuroscience community and are often seen as a "microcosm of object recognition". In particular, we wanted to compare the responses of single cortical neurons in the face patches at the end of the visual processing hierarchy, recorded by our collaborators to a recently emerged class of so called "disentangling" deep neural networks that, unlike the usual "black box" systems, explicitly aim to be interpretable to humans. A "disentangling" neural network learns to map complex images into a small number of internal neurons (called latent units), each one representing a single semantically meaningful attribute of the scene, like colour or size of an object (see Figure 1). Unlike the "black box" deep classifiers trained to recognise visual objects through a biologically unrealistic amount of external supervision, such disentangling models are trained without an external teaching signal using a self-supervised objective of reconstructing input images (generation in Figure 1) from their learnt latent representation (obtained through inference in Figure 1).