GLMs Part III: Deep Neural Networks as Recursive Generalized Linear Models

Generalized Linear Models (GLMs) play a critical role in fields including Statistics, Data Science, Machine Learning, and other computational sciences. Part I of this Series provided a thorough mathematical overview with proofs of common GLMs, both in Canonical and Non-Canonical forms. Part II provided historical and mathematical context of common iterative numerical fitting procedures for GLMs including Newton-Raphson, Fisher Scoring, Iteratively Reweighted Least Squares, and Gradient Descent. In the last of this three-part Series, we explore Neural Networks and their connection with GLMs. In-fact, Neural Networks are nothing more than recursive Canonical GLMs.