Mean-field theory of input dimensionality reduction in unsupervised deep neural networks

This is achieved by creating progressively better representations of sensory inputs, and these representations finally become easily-decoded without any reward or supervision signals [1-3]. This kind of learning is called unsupervised learning, which has long been thought of as a fundamental function of the sensory cortex [4]. Based on the similar computational principle, many layers of artificial neural networks were designed to perform a nonlinear dimensionality reduction of high dimensional data [5], which later triggered resurgence of deep neural networks. By stacking unsupervised modules on top of each other, one can produce a deep feature hierarchy, in which high-level features can be constructed from less abstract ones along the hierarchy. However, it remains rarely explored how this kind of effective representation is transformed along stages of processing.