Automatic differentiation for machine learning in Julia - Julia language blog

Sequence of functions above is derived from expression graph of our input function \(f\) – it decomposes our function into sequence of functions we know how to handle. Now our function is a sequence of basic operations that change variables' values. Forward mode automatic differentiation reduces to computing partial derivative with respect to chosen input dimension at given point by differentiating each of the sequence elements forward. Lets try with point \((3,5)\).