Mathematics of Neural Network

Here, α is the learning rate. Using this we can control the rate at which the weights get trained and updated. In order to perform the partial differentiation, we need to understand the Chain Rule. In the diagram above, the highlighted part shows the flow of information, from last layer to the Layer # 5. We know that, back propagation helps us in optimise/update weights and biases by using the below relation: Now, we already have the weights with us, usually we decide α using hyperparameter tuning. The only part left to calculate is the partial differentiation term.