Building an autodifferentiation library – Maciej Kula – Medium

Popular general-purpose auto-differentiation frameworks like PyTorch or TensorFlow are very capable, and, for the most part, there is little need for writing something more specialized. Nevertheless, I have recently started writing my own autodiff package. This blog post describes what I've learned along the way. Think of this as a poor-man's version of a Julia Evans blog post. Note that there are many blog posts describing the mechanics of autodifferentiation much better than I could, so I skip the explanations here.