10 questions about deep learning

It seems everywhere you look nowadays, you will find an article that describes a winning strategy using deep learning in a data science problem, or more specifically in the field of artificial intelligence (AI). However, clear explanations of deep learning, why it's so powerful, and the various forms deep learning takes in practice, are not so easy to come by. In order to know more about deep learning, neural networks, the major innovations, the most widely used paradigms, where deep learning works and doesn't, and even a little of the history, we have asked and answered a few basic questions. Deep learning is the modern evolution of traditional neural networks. Indeed, to the classic feed-forward, fully connected, backpropagation trained, multilayer perceptrons (MLPs), "deeper" architectures have been added.