Latent Gaussian models (LGMs) are a popular class of Bayesian hierarchical models that include Gaussian processes, as well as certain spatial models and mixed-effect models. Efficient Bayesian inference of LGMs often requires marginalizing out the latent variables. For LGMs with a non-Gaussian likelihood, exact marginalization is not possible and a popular approach is to do approximate marginalization with an integrated Laplace approximation (ILA). Using ILA produces an approximate posterior which, in some settings, can differ significantly from the correct posterior, which impacts downstream applications. We propose an importance sampling scheme to correct the error introduced by ILA. By increasing the number of samples in importance sampling, the posterior with ILA converges to the correct posterior. This idea is realized with various techniques, including pseudo-marginalization, quasi-Monte Carlo and randomized quasi-Monte Carlo. We implement our methods in an automatic differentiation framework to support gradient-based algorithms when doing inference on the hyperparameters. For the latter, we specifically consider the use of Hamiltonian Monte Carlo. We demonstrate the benefits of reduced error in various applied models.

When the activation function of a network is analytic (e.g., sigmoid

Neural Information Processing Systems http://nips.cc/

Its derivation, based on Lu et al.[54] is as follows. For the HMC baseline, we use the default implementation of NUTS in Pyro. In Table 7, we present the detailed, non-averaged results to complement Table 4. In both cases, we observe that the performance of the refined posterior approaches HMC's. C.2 Textclassification We further validate the proposed method on text classification problems.

Markov chain Monte Carlo (MCMC) is an inference mechanism that approximates a target probability distribution by a sequence of states (a.k.a.

Table evaluated hyperparameters Dataset Nd GPR |M| - - q() - - free-form Boston 506 13 3.049 Concrete 1030 8 4.864 Ener 768 8 0.441 WineRed1599 11 0.640 Yacht308 6 0.353