Making Backpropagation, Autograd, MNIST Classifier from scratch in Python

Backpropagation (backward propagation of errors) -- is a widely used algorithm in training feedforward networks. It computes the gradient of the loss function with respect to the weights of the network. The main idea of it is to break big functions in small parts and use partial derivatives to get function derivative with using the Chain Rule. And because solving this can be a very hard task, here comes backpropagation and gradient descent(updating weights by a small amount based on the gradient to move in the way of loss minimization). Let's say we have 3 variables x -2, y 5, z -4, the result will be f -12, and our target for training is -13.