Neural networks and deep learning

Why are deep neural networks hard to train? Appendix: Is there a simple algorithm for intelligence? If you benefit from the book, please make a small donation. I suggest $5, but you can choose the amount. Thanks to all the supporters who made the book possible, with especial thanks to Pavel Dudrenov. In the last chapter we learned that deep neural networks are often much harder to train than shallow neural networks. That's unfortunate, since we have good reason to believe that if we could train deep nets they'd be much more powerful than shallow nets. But while the news from the last chapter is discouraging, we won't let it stop us. In this chapter, we'll develop techniques which can be used to train deep networks, and apply them in practice. We'll also look at the broader picture, briefly reviewing recent progress on using deep nets for image recognition, speech recognition, and other applications. And we'll take a brief, speculative look at what the future may hold for neural nets, ...