Learning with hidden variables

Learning and inferring features that generate sensory input is a task continuously performed by cortex. In recent years, novel algorithms and learning rules have been proposed that allow neural network models to learn such features from natural images, written text, audio signals, etc. These networks usually involve deep architectures with many layers of hidden neurons. Here we review recent advancements in this area emphasizing, amongst other things, the processing of dynamical inputs by networks with hidden nodes and the role of single neuron models. These points and the questions they arise can provide conceptual advancements in understanding of learning in the cortex and the relationship between machine learning approaches to learning with hidden nodes and those in cortical circuits. Keywords: statistical models, deep learning, dynamics 1. Introduction Learning the fundamental features that generate sensory signals and having the ability to infer these features once given the sensory input is a crucial aspect of information processing. It would not be far fetched to hypothesize that the organization of cortical circuitry is largely evolved for performing such tasks, or to put it slightly differently, that our evolutionary history has stored memories of such features in the connections that form a large part of our brains. But how can a neuronal network learn and extract features that cause the experiences of our sensory organs, in a supervised or unsupervised manner?