An overview on gradient descent and its variants

The term "optimization" refers to the process of iteratively training a model to produce a maximum and minimum function evaluation to get a minimum cost function. It is crucial since it will assist us in obtaining a model with the least amount of error (as there will be discrepancies between the actual and predicted values). There are various optimization methods; in this article, we'll look at gradient descent and its three forms: batch, stochastic, and mini-batch. Note: Hyperparameter optimization is required to fine-tune the model. Before you begin training the model, you must first specify hyperparameters.