Recommendations for Deep Learning Neural Network Practitioners

Deep learning neural networks are relatively straightforward to define and train given the wide adoption of open source libraries. Nevertheless, neural networks remain challenging to configure and train. In his 2012 paper titled "Practical Recommendations for Gradient-Based Training of Deep Architectures" published as a preprint and a chapter of the popular 2012 book "Neural Networks: Tricks of the Trade," Yoshua Bengio, one of the fathers of the field of deep learning, provides practical recommendations for configuring and tuning neural network models. In this post, you will step through this long and interesting paper and pick out the most relevant tips and tricks for modern deep learning practitioners. Practical Recommendations for Deep Learning Neural Network Practitioners Photo by Susanne Nilsson, some rights reserved.