3 Thoughts on Why Deep Learning Works So Well

Last week, deep learning research leader Yann LeCun took part in a Quora Session, during which he answered questions from community members on a wide variety of (mostly machine/deep learning) topics. When will we see a theoretical background and mathematical foundation for deep learning? The answer turned into a very eloquent overview of three particular thoughts on why deep learning works so well. Here is a quick overview. LeCun's first point of explanation, which maps to a good reason why deep learning works so well, is as follows: One theoretical puzzle is why the type of non-convex optimization that needs to be done when training deep neural nets seems to work reliably.