Yann LeCun is among those bringing a new level of artificial intelligence to popular internet services from the likes of Facebook, Google, and Microsoft. As the head of AI research at Facebook, LeCun oversees the creation of vast "neural networks" that can recognize photos and respond to everyday human language. And similar work is driving speech recognition on Google's Android phones, instant language translation on Microsoft's Skype service, and so many other online tools that can "learn" over time. Using vast networks of computer processors, these systems approximate the networks of neurons inside the human brain, and in some ways, they can outperform humans themselves. This week in the scientific journal Nature, LeCun--also a professor of computer science at New York University--details the current state of this "deep learning" technology in a paper penned alongside the two other academics most responsible for this movement: University of Toronto professor Geoff Hinton, who's now at Google, and the University of Montreal's Yoshua Bengio.
The four founders, two of whom have resigned their professorships to devote their full attention to TwentyBN, met each other during studies at the University of Bielefeld in Germany. Each of the founders has over 15 years of experience in machine learning and the relatively young deep learning discipline. Prof. Dr. Roland Memisevic, Chief Scientist, received his doctorate in Toronto, studying with Geoffrey Hinton, one of the founding fathers of deep learning. Prior to co-founding Twenty Billion Neurons, Memisevic was a member of the faculty at the renowned Machine Learning Institute of the University of Montréal led by Yoshua Bengio. The institute counts Google, Facebook, and IBM amongst its most active donors.
The CIFAR deep learning summer school in Toronto has been training the top AI researchers entering or finishing Ph.D. programs since 2005. Over 1,200 students from 60 different countries applied, of which 200 were selected to attend. Attendees represent some of the leading AI labs in the world, Montreal Institute of Learning Algorithms (MILA), University College London, University of Toronto, University of Alberta, Berkeley, NYU, Columbia, CMU, MIT, ETH Zurich, and Stanford. Every year, the school has trained the next generation of top AI researchers which now hold top posts at AI companies like Google, Facebook, Tesla, and Uber. During an intense 10-day period, students learn the tricks of the trade from top AI researchers like deep learning pioneers Yoshua Bengio (MILA), Geoff Hinton (UofT), and reinforcement learning pioneer, Richard Sutton (University of Alberta, Google Deepmind).
Thirty years ago, Yann LeCun pioneered the use of a particular form of machine learning, called the convolutional neural network, or CNN, while at the University of Toronto. That approach, moving a filter over a set of pixels to detect patterns in images, showed promise in cracking problems such as getting the computer to recognize hand-written digits with minimal human guidance. Years later, LeCun, then at NYU, launched a "conspiracy," as he has termed it, to bring machine learning back into the limelight after a long winter for the discipline. The key was LeCun's CNN, which had continued to develop in sophistication to the point where it could produce results in computer vision that stunned the field. The new breakthroughs with CNNs, along with innovations by peers such as Yoshua Bengio, of Montreal's MILA group for machine learning, and Geoffrey Hinton of Google Brain, succeeded in creating a new springtime for AI research, in the form of deep learning.
The research lab will be led by Maluuba's CTO, Kaheer Suleman, and will be staffed by 13 deep learning research scientists. Maluuba has also partnered with reinforcement learning expert Richard Sutton, a principal investigator from the Alberta Innovates Centre for Machine Learning and an Association for the Advancement of Artificial Intelligence Fellow. "Maluuba is working with leading experts and the world's premiere academic centre for deep learning to design systems that can represent knowledge and answer questions in natural language. The potential applications of this research are staggering." The company counts LG as one of its customers.