Backpropagation Chain Rule

The chain rule is a fundamental result in calculus. Besides being a handy tool for computing derivatives in calculus homework, the chain rule is closely related to the backpropagation algorithm that is widely-used for computing derivatives (gradients) in neural network training. This blog post by Boaz Barak is a beautiful tutorial on the chain rule and the backpropagation algorithm. As in Barak's post, the backpropagation algorithm is usually taught as an application of the chain rule in machine learning classes. This leads to a common belief that "backpropagation is just applying the chain rule repeatedly".