Consequently, there are no theoretical guarantees so far for this class of methods.

This paper addresses the challenging computational problem of estimating intractable expectations over discrete domains. Existing approaches, including Monte Carlo and Russian Roulette estimators, are consistent but often require a large number of samples to achieve accurate results. We propose a novel estimator, \emph{BayesSum}, which is an extension of Bayesian quadrature to discrete domains. It is more sample efficient than alternatives due to its ability to make use of prior information about the integrand through a Gaussian process. We show this through theory, deriving a convergence rate significantly faster than Monte Carlo in a broad range of settings. We also demonstrate empirically that our proposed method does indeed require fewer samples on several synthetic settings as well as for parameter estimation for Conway-Maxwell-Poisson and Potts models.

We introduce here a novel sample-efficient inference framework, V ariational Bayesian Monte Carlo (VBMC). VBMC combines variational inference with Gaussian-process based, active-sampling Bayesian quadrature, using the latter to efficiently approximate the intractable integral in the variational objective.

Neural Information Processing Systems http://nips.cc/

The accuracy of a quadrature rule is assessed by the quadrature error, i.e., the absolute difference between the left-hand side and the right-hand side of

Calculation of Bayesian posteriors and model evidences typically requires numerical integration.

Now, since as mentioned above the KL divergence is a non-negative quantity, from Eq. S2 we have

We introduce new'global' acquisition functions, such as expected

First provide a summary of the paper, and then address the following criteria: Quality, clarity, originality and significance. Overview: this paper presents a fast alternative to MC methods for approximating intractable integrals. The main idea behind Bayesian quadrature is to exploit assumptions and regularities in the likelihood surface, something which pure Monte Carlo ignores. Samples are then drawn according to some criterion - in this case, samples are chosen to the location of the maximal expected posterior variance of the integrand. Intuitively, this is a location where the model knows the least about the value of the integrand, and stands to gain a lot of information.

Now, since as mentioned above the KL divergence is a non-negative quantity, from Eq. S2 we have