Tensor-Based Foundations of Ordinary Least Squares and Neural Network Regression Models

This article introduces a novel approach to the mathematical development of Ordinary Least Squares and Neural Network regression models, diverging from traditional methods in current Machine Learning literature. By leveraging Tensor Analysis and fundamental matrix computations, the theoretical foundations of both models are meticulously detailed and extended to their complete algorithmic forms. The study culminates in the presentation of three algorithms, including a streamlined version of the Backpropagation Algorithm for Neural Networks, illustrating the benefits of this new mathematical approach. The following sections of this article require some important mathematical concepts and notations that need to be addressed here in this section. However, we assume the reader to be proficient on the basic and intermediate topics of Linear Algebra and Calculus, which will not be covered.