Bayesian Learning
Frank-Wolfe Bayesian Quadrature: Probabilistic Integration with Theoretical Guarantees
François-Xavier Briol, Chris Oates, Mark Girolami, Michael A. Osborne
There is renewed interest in formulating integration as a statistical inference problem, motivated by obtaining a full distribution over numerical error that can be propagated through subsequent computation. Current methods, such as Bayesian Quadrature, demonstrate impressive empirical performance but lack theoretical analysis. An important challenge is therefore to reconcile these probabilistic integrators with rigorous convergence guarantees. In this paper, we present the first probabilistic integrator that admits such theoretical treatment, called Frank-Wolfe Bayesian Quadrature (FWBQ). Under FWBQ, convergence to the true value of the integral is shown to be up to exponential and posterior contraction rates are proven to be up to super-exponential. In simulations, FWBQ is competitive with state-of-the-art methods and out-performs alternatives based on Frank-Wolfe optimisation. Our approach is applied to successfully quantify numerical error in the solution to a challenging Bayesian model choice problem in cellular biology.
A Recurrent Latent Variable Model for Sequential Data
Junyoung Chung, Kyle Kastner, Laurent Dinh, Kratarth Goel, Aaron C. Courville, Yoshua Bengio
Neural Information Processing Systems http://nips.cc/
DAGs with No Fears: A Closer Look at Continuous Optimization for Learning Bayesian Networks
We first generalize existing algebraic characterizations of acyclicity to a class of matrix polynomials.
Poisson-Minibatching for Gibbs Sampling with Convergence Rate Guarantees
Ruqi Zhang, Christopher M. De Sa
Gibbs sampling is a Markov chain Monte Carlo method that is often used for learning and inference on graphical models.
Rapidly Mixing Gibbs Sampling for a Class of Factor Graphs Using Hierarchy Width
Christopher M. De Sa, Ce Zhang, Kunle Olukotun, Christopher None Ré
Neural Information Processing Systems http://nips.cc/
Synaptic Sampling: A Bayesian Approach to Neural Network Plasticity and Rewiring
David Kappel, Stefan Habenschuss, Robert Legenstein, Wolfgang Maass
Neural Information Processing Systems http://nips.cc/
On the Convergence of Stochastic Gradient MCMC Algorithms with High-Order Integrators
Changyou Chen, Nan Ding, Lawrence Carin
Recent advances in Bayesian learning with large-scale data have witnessed emergence of stochastic gradient MCMC algorithms (SG-MCMC), such as stochastic gradient Langevin dynamics (SGLD), stochastic gradient Hamiltonian MCMC (SGHMC), and the stochastic gradient thermostat. While finite-time convergence properties of the SGLD with a 1st-order Euler integrator have recently been studied, corresponding theory for general SG-MCMCs has not been explored. In this paper we consider general SG-MCMCs with high-order integrators, and develop theory to analyze finite-time convergence properties and their asymptotic invariant measures. Our theoretical results show faster convergence rates and more accurate invariant measures for SG-MCMCs with higher-order integrators.
Globally optimal score-based learning of directed acyclic graphs in high-dimensions
Bryon Aragam, Arash Amini, Qing Zhou
This estimator has been the subject of extensive empirical inquiry [e.g.
Debiased Bayesian inference for average treatment effects
Neural Information Processing Systems http://nips.cc/