Visualizing the gradient descent method

In the gradient descent method of optimization, a hypothesis function, h_\boldsymbol{\theta}(x), is fitted to a data set, (x {(i)}, y {(i)}) ( i 1,2,\cdots,m) by minimizing an associated cost function, J(\boldsymbol{\theta}) in terms of the parameters \boldsymbol\theta \theta_0, \theta_1, \cdots . The cost function describes how closely the hypothesis fits the data for a given choice of \boldsymbol \theta . For example, one might wish to fit a given data set to a straight line, h_\boldsymbol{\theta}(x) \theta_0 \theta_1 x. To simplify things, consider fitting a data set to a straight line through the origin: h_\theta(x) \theta_1 x . In this one-dimensional problem, we can plot a simple graph for J(\theta_1) and follow the iterative procedure which trys to converge on its minimum.