Stochastic Gradient Descent - Mini-batch and more - Adventures in Machine Learning

In the neural network tutorial, I introduced the gradient descent algorithm which is used to train the weights in an artificial neural network. In reality, for deep learning and big data tasks standard gradient descent is not often used. Rather, a variant of gradient descent called stochastic gradient descent and in particular its cousin mini-batch gradient descent is used. That is the focus of this post. The gradient descent optimisation algorithm aims to minimise some cost/loss function based on that function's gradient.