Using the Equivalent Kernel to Understand Gaussian Process Regression

The equivalent kernel [1] is a way of understanding how Gaussian pro- cess regression works for large sample sizes based on a continuum limit. In this paper we show (1) how to approximate the equivalent kernel of the widely-used squared exponential (or Gaussian) kernel and related ker- nels, and (2) how analysis using the equivalent kernel helps to understand the learning curves for Gaussian processes. Consider the supervised regression problem for a dataset D with entries (xi, yi) for i 1, . . . We can define a vector of functions h(x) (K 2I)-1k(x) . Thus we have f (x) h (x)y, making it clear that the mean prediction at a point x is a linear combination of the target values y.