What is Gradient Descent?

Gradient Descent is a popular optimization technique where the general idea is to tweak(adjusting till we get optimal result) parameters iteratively in order to minimize the cost function. It measures the local gradient of the error function with respect to the parameter vector θ, and it goes in the direction of the descending gradient. Once the gradient is zero, you have reached a minimum. Gradient Descent is useful when you have a very large dataset. So the process is, you will start by filling θ with random values, this is called random initialization, and then you improve it gradually, taking one tiny step at a time, at each step you are attempting to decrease the cost function until the algorithm converges to a minimum.