Supplementary Informationfor: FastMatrixSquare RootswithApplicationstoGaussianProcessesand BayesianOptimization

We note that all methods incur some sampling error, regardless of the subset size (N). In Fig. S6 we plot the learned hyperparameters of the Precipitation SVGP models: 1)o2 (the kernel outputscale)--which roughly corresponds to variance explained as "signal" in the data; 2)σ2obs--which roughly corresponds to variance explained away as observational noise; and 3)ν (degreesoffreedom)--which controls thetailsofthenoisemodel (lowerν corresponds toheavier tails). As M increases, we find that the observational noise parameter decreases by a factor of 4--downfrom 0.19to0.05--whilethe Fig. S7 is a histogram displaying the msMINRES iterations needed to achieve a relative residual of10 3 when training aM = 5,000SVGP model on the 3droad dataset (subsampled to30,000 datapoints). AsM increases, the kernel outputscale (left) also increases.