95c7dfc5538e1ce71301cf92a9a96bd0-Supplemental.pdf

For regression, we model output noise as a zero-mean Gaussian: N(0,σ2) where σ2 is the varianceofthenoise,treatedasahyperparameter. Neal[21] shows that in the regression setting, the isotropic Gaussian prior for a BNN with a single hidden layer approaches aGaussian process prior asthe number ofhidden units tends toinfinity,solong as the chosen activation function is bounded. We will use this prior in the baseline BNN for our experiments. In the context of BNNs, our Markov chain is a sequence ofrandomparametersW(1),W(2),... definedoverW,whichweconstruct bydefining thetransitionkernel. BBB is scalable and fast, and therefore can be applied to high-dimensional and large datasets in real-life applications.