How does PyTorch calculate gradient: a programming perspective

PyTorch uses the autograd package for automatic differentiation. For a tensor y, we can calculate the gradient with respect to input with two methods. After we do the .backward(), PyTorch generates a Dynamic Computation Graph when the forward function of network is called. We borrow a toy example from here.