"E-step" (Steps 1 and 2 in Algorithm 1) and a weighted maximum likelihood estimation (MLE) "M-step" (Step 3; see [ ( t 1) (t 1) One may use these in a number of different ways. The following observation is due to Chebyshev's inequality. One can use Proposition S2.1 to construct a confidence interval on, for example, the expected squared Note that 1) the bound in Proposition S2.1 is CbAS naturally controls the importance weight variance. Design procedures that leverage a trust region can naturally bound the variance of the importance weights. We used CbAS as follows.

Currently there are several theories for how neural activity may represent probability distributions.

In practice, however, recursive updating often leads to poor trade-off solutions across tasks because approximate inference is necessary for most models of interest.

It is often convenient to model high-dimensional datasets as probability distributions.

We will often use variational optimization. When the variational family is not made explicit it means that the optimization is over all functions with the correct domain and codomain, e.g.

The prior and likelihood are both modelling choices. A.1 Likelihoods for BNNs The likelihood is purely a function of the model prediction Φ As exact posterior inference via (11) is intractable, we instead rely on approximate inference algorithms, which can be broadly grouped into two classes based on their method of approximation. A concrete label can be obtained by choosing the class with highest output value. The Gaussian variational family is a common choice. Estimators for the integral in (15) are necessary.

Work done while at Harvard University.

Undirected graphical models are compact representations of joint probability distributions over random variables. To solve inference tasks of interest, graphical models of arbitrary topology can be trained using empirical risk minimization.