Rényi Divergence Variational Inference Richard E. Turner University of Cambridge University of Cambridge Cambridge, CB2 1PZ, UK Cambridge, CB2 1PZ, UK yl494@cam.ac.uk ret26@cam.ac.uk

This new family of variational methods unifies a number of existing approaches, and enables a smooth interpolation from the evidence lower-bound to the log (marginal) likelihood that is controlled by the value of α that parametrises the divergence. The reparameterization trick, Monte Carlo approximation and stochastic optimisation methods are deployed to obtain a tractable and unified framework for optimisation. We further consider negative α values and propose a novel variational inference method as a new special case in the proposed framework. Experiments on Bayesian neural networks and variational auto-encoders demonstrate the wide applicability of the VR bound.