Optimizers of the neural network

An optimizer is a function or an algorithm that modifies the attributes of the neural network, such as weights and learning rate, it also reduces the loss (loss can be calculated between the predicted value and actual value) and improves the accuracy. It takes an entire dataset for each iteration which consumes more time and there might be a chance of skipping the optimal value. SGD updates the weights after seeing each data point instead of the dataset. Momentum: it helps to move faster towards the loss function. Adding a fraction of the previous update to the current update will make the process a bit faster.