The Matrix Calculus You Need For Deep Learning (Notes from a paper by Terence Parr and Jeremy… - DEV Community

Jeremy's courses show how to become a world-class deep learning practitioner with only a minimal level of scalar calculus, thanks to leveraging the automatic differentiation built in to modern deep learning libraries. But if you really want to really understand what's going on under the hood of these libraries, and grok academic papers discussing the latest advances in model training techniques, you'll need to understand certain bits of the field of matrix calculus. Hopefully you remember some of these main scalar derivative rules. If your memory is a bit fuzzy on this, have a look at Khan academy video on scalar derivative rules. There are other rules for trigonometry, exponential, etc., which you can find at Khan Academy differential calculus course.