Cost (Loss) Function in Machine Learning

Although there are other variants of cost function as mentioned at the very beginning by saying different variations (see MAE, RMSE, MSE), in this article we will consider the squared error function, which is one of the cost calculation functions and also is effective to use for many regression problems. Since the aim is to find the most accurate model, our main goal is to minimize the cost function, that is, the error. As seen in this image, we should use the optimal theta values of the J cost function, which are the theta values of the point where the error is minimum, in the model. To show it correctly in 2D, let's consider the function simplified, that is, theta zero value (constant) is 0. As can be seen in the figure, we start the calculation by accepting (randomly) the theta 1 value as 0.5. When we calculate the error, we get the value of approximately 0.58 and so, mark the point (0.5, 0.58) on the graph.