Why Gradient Descent Works?

Gradient descent is an iterative optimization algorithm that is used to optimize the weights of a machine learning model (linear regression, neural networks, etc.) by minimizing the cost function of that model. The intuition behind gradient descent is this: Picture the cost function (denoted by f(Θ) where Θ [Θ₁, … Θₙ]) plotted in n dimensions as a bowl. Imagine a randomly placed point on that bowl represented by n coordinates (this is the initial value of your cost function). The minimum of this "function" then will be the bottom of the bowl. The goal is then to reach to the bottom of the bowl (or minimize the cost) by progressively moving downwards on the bowl.