Gradient Descent with 'Math'

In the last blog we saw about basics of Gradient Descent and how it works.This time we will see math behind it. We are actually subtracting some part from value of parameter and updating it.We keep doing this until we get optimized value of parameter so the cost is minimum. You may be thinking that why '-' sign is used in above equation. If you look at image below, in the right side of curve slope is positive so by subtracting value from theta, we are actually getting closer to the optimal value, while on the left side the slope is negative so we are actually adding some part in value of theta and so getting closer to the optimal value. We keep updating value of theta until the change in value 0.001 (values may vary according to case).Usually we take value of learning rate as 0.01