CNNs, like visual cortex, build a representation of the visual world that is useful to the "viewer". We have known for a while that CNNs trained on object recognition tasks capture some (but not all) aspects of the representation computed by primate visual cortex. Here the authors propose to bridge the gap by explicitly encouraging a CNN to build a representation that is "similar" to the one computed by the visual cortex of mice. This is a neat idea and certainly a novel one. The paper is clearly written, which I appreciated.

For something so effortless and automatic, vision is a tough job for the brain. It's remarkable that we can transform electromagnetic radiation--light--into a meaningful world of objects and scenes. After all, light focused into an eye is merely a stream of photons with different wave properties, projecting continuously on our retinas, a layer of cells on the backside of our eyes. Before it's transduced by our eyes, light has no brightness or color, which are properties of animal perception. Our retinas transform this energy into electrical impulses that propagate within our nervous system. Somehow this comes out as a world: skies, children, art, auroras, and occasionally ghosts and UFOs.

Deep learning systems are revolutionizing technology around us, from voice recognition that pairs you with your phone to autonomous vehicles that are increasingly able to see and recognize obstacles ahead. But much of this success involves trial and error when it comes to the deep learning networks themselves. A group of MIT researchers recently reviewed their contributions to a better theoretical understanding of deep learning networks, providing direction for the field moving forward. "Deep learning was in some ways an accidental discovery," explains Tommy Poggio, investigator at the McGovern Institute for Brain Research, director of the Center for Brains, Minds, and Machines (CBMM), and the Eugene McDermott Professor in Brain and Cognitive Sciences. "We still do not understand why it works. A theoretical framework is taking form, and I believe that we are now close to a satisfactory theory. It is time to stand back and review recent insights."

Deep learning systems are revolutionizing technology around us, from voice recognition that pairs you with your phone to autonomous vehicles that are increasingly able to see and recognize obstacles ahead. But much of this success involves trial and error when it comes to the deep learning networks themselves. A group of MIT researchers recently reviewed their contributions to a better theoretical understanding of deep learning networks, providing direction for the field moving forward. "Deep learning was in some ways an accidental discovery," explains Tommy Poggio, investigator at the McGovern Institute for Brain Research, director of the Center for Brains, Minds, and Machines (CBMM), and the Eugene McDermott Professor in Brain and Cognitive Sciences. "We still do not understand why it works. A theoretical framework is taking form, and I believe that we are now close to a satisfactory theory. It is time to stand back and review recent insights."

Deep learning systems are revolutionizing technology around us, from voice recognition that pairs you with your phone to autonomous vehicles that are increasingly able to see and recognize obstacles ahead. But much of this success involves trial and error when it comes to the deep learning networks themselves. A group of MIT researchers recently reviewed their contributions to a better theoretical understanding of deep learning networks, providing direction for the field moving forward. "Deep learning was in some ways an accidental discovery," explains Tommy Poggio, investigator at the McGovern Institute for Brain Research, director of the Center for Brains, Minds, and Machines (CBMM), and the Eugene McDermott Professor in Brain and Cognitive Sciences. "We still do not understand why it works. A theoretical framework is taking form, and I believe that we are now close to a satisfactory theory. It is time to stand back and review recent insights."

Introductory statistics courses teach us that, when fitting a model to some data, we should have more data than free parameters to avoid the danger of overfitting -- fitting noisy data too closely, and thereby failing to fit new data. It is surprising, then, that in modern deep learning the practice is to have orders of magnitude more parameters than data. Despite this, deep networks show good predictive performance, and in fact do better the more parameters they have. It has been known for some time that good performance in machine learning comes from controlling the complexity of networks, which is not just a simple function of the number of free parameters. The complexity of a classifier, such as a neural network, depends on measuring the "size" of the space of functions that this network represents, with multiple technical measures previously suggested: Vapnik–Chervonenkis dimension, covering numbers, or Rademacher complexity, to name a few.

Chethan Pandarinath wants to enable people with paralysed limbs to reach out and grasp with a robotic arm as naturally as they would their own. To help him meet this goal, he has collected recordings of brain activity in people with paralysis. His hope, which is shared by many other researchers, is that he will be able to identify the patterns of electrical activity in neurons that correspond to a person's attempts to move their arm in a particular way, so that the instruction can then be fed to a prosthesis. Essentially, he wants to read their minds. "It turns out, that's a really challenging problem," says Pandarinath, a biomedical engineer at the Georgia Institute of Technology in Atlanta. "These signals from the brain -- they're really complicated."

Over the past decade, advances in deep learning have transformed the fortunes of the artificial intelligence (AI) community. The neural network approach that researchers had largely written off by the end of the 1990s now seems likely to become the most widespread technology in machine learning. However, protagonists find it difficult to explain why deep learning often works well, but is prone to seemingly bizarre failures. The success of deep learning came with rapid improvements in computational power that came through the development of highly parallelized microprocessors and the discovery of ways to train networks with enormous numbers of virtual neurons assembled into tens of linked layers. Before these advances, neural networks were limited to simple structures that were easily outclassed in image and audio classification tasks by other machine-learning architectures such as support vector machines.

The problem of learning is arguably at the very core of the problem of intelligence, both biological and artificial. In this article, we review our work over the last 10 years in the area of supervised learning, focusing on three interlinked directions of research--(1) theory, (2) engineering applications (making intelligent software), and (3) neuroscience (understanding the brain's mechanisms of learnings)--that contribute to and complement each other. Because seeing is intelligence, learning is also becoming a key to the study of artificial and biological vision. In the last few years, both computer vision--which attempts to build machines that see--and visual neuroscience--which aims to understand how our visual system works--are undergoing a fundamental change in their approaches. Visual neuroscience is beginning to focus on the mechanisms that allow the cortex to adapt its circuitry and learn a new task.

The entire field of neural network integrates neuroscience and computer science together and holds staggering potentials in this area of data analysis. Neural network emerged in the mid 1940's and was investigated under the aforementioned fields up until the late 1960's. Neural networks have the incredible capacity to provide tangible information out of some of the complicated and inaccurate data. While the field was abandoned for a while, the last few decades has witnessed the resurgence of research into the field of neural networks which has been characterized by tremendous funding of the field and major advancements. In the simplest form, neural networks can be explained as a way to substitute manual engineering analytical operations of mechanisms for a different mechanism operation in which mechanism are taught to analyze information like the brain and nervous system would.