Statistical Learning
Variational Inference with Tail-adaptive f-Divergence
Dilin Wang, Hao Liu, Qiang Liu
V ariational inference (VI) (e.g., Jordan et al., 1999; Wainwright et al., 2008) has been established Combined with techniques like stochastic optimization (Ranganath et al., A key component of successful variational inference lies on choosing a proper divergence metric. Work done at UT Austin 32nd Conference on Neural Information Processing Systems (NeurIPS 2018), Montrรฉal, Canada.
Exponentiated Strongly Rayleigh Distributions
Strongly Rayleigh (SR) measures are discrete probability distributions over the subsets of a ground set. They enjoy strong negative dependence properties, as a result of which they assign higher probability to subsets of diverse elements. We introduce in this paper Exponentiated Strongly Rayleigh (ESR) measures, which sharpen (or smoothen) the negative dependence property of SR measures via a single parameter (the exponent) that can be intuitively understood as an inverse temperature. We develop efficient MCMC procedures for approximate sampling from ESRs, and obtain explicit mixing time bounds for two concrete instances: exponentiated versions of Determinantal Point Processes and Dual V olume Sampling. We illustrate some of the potential of ESRs, by applying them to a few machine learning problems; empirical results confirm that beyond their theoretical appeal, ESR-based models hold significant promise for these tasks.