A Approximate Inference

This section provides further details on the algorithms introduced in Section 3.1. BBB [7] is an application of VI to deep neural network. ELBO by itself is not differentiable as it depends on the randomly chosen parameters. However, the reparameterization trick [44] applies to diagonal Gaussians and allows us to use the negative ELBO as the loss function. While there have been reports of BBB performing well when used on neural networks [7, 86], the current consensus of the research community seems to be that BBB falls short when compared to e.g.