Gaussian Process Regression From First Principles

In this article, we'll discuss Gaussian Process Regression (GPR) from first principles, using mathematical concepts from machine learning, optimization, and Bayesian inference. We'll start with Gaussian Processes, use this to formalize how predictions are made with GPR models, and then discuss two crucial ingredients for GPR models: covariance functions and hyperparameter optimization. Finally, we'll build on our mathematical derivations below by discussing some intuitive ways to view GPR. If you'd also like to see these ideas presented as an academic-style paper, please check out this link here. Before we talk about GPR, let's first explore what a Gaussian Process is.