Supplementary Material: Appendix Bayesian Deep Ensembles via the Neural Tangent Kernel A Recap of standard and NTK parameterisations

Sohl-Dickstein et al. [32] have recently explored further variants of these parameterisations. Note that Eq. (29) requires computation of two forward passes Eq. (29) may have similar costs to our main construction in Section 3, for certain AD packages. As stated in Lemma 3 of Osband et al. Thus, solving Eq. (31) will lead to the same functional outputs in both parameterisations, if the NN remains in the linearised regime. Here, we present our ensemble algorithms for NTKGP-Lin (Algorithm 2) and NTKGP-fn (Algorithm 3), to complement the NTKGP-param algorithm that was presented in Section 3.4. For completeness, we now describe how to aggregate predictions from ensemble members.