A Gentle Introduction to Exploding Gradients in Neural Networks - Machine Learning Mastery

Exploding gradients are a problem where large error gradients accumulate and result in very large updates to neural network model weights during training. This has the effect of your model being unstable and unable to learn from your training data. In this post, you will discover the problem of exploding gradients with deep artificial neural networks. A Gentle Introduction to Exploding Gradients in Recurrent Neural Networks Photo by Taro Taylor, some rights reserved. An error gradient is the direction and magnitude calculated during the training of a neural network that is used to update the network weights in the right direction and by the right amount.