A Supplementary Material

A.1 Proof of Propostion 1 We prove that the optimal response-linear achievable interpolator in linear regression is w Claim 1. (18) implies w Lebesgue measure, (21) is hence a contradiction to (20). A.4 Assumption 1 implies rank (X) = n with probability 1. The proof uses techniques which were already developed in Hastie et al. (2019). However, to fully finish the proof, one needs to first justify exchanging the limits in (26). In this subsection we justify the statement (of the last paragraph of Section 5.3) that, in the strong The off-diagonal entries are 0. 20 Figure 4: Plot of It is interesting to study which covariance matrix approximators one should use. However, the considered examples used an isotropic prior, i.e.