Uncertainty
f-Divergence Variational Inference
V ariational inference (VI) is a machine learning method that makes Bayesian inference computationally efficient and scalable to large datasets.
Advances in Black-Box VI: Normalizing Flows, Importance Weighting, and Optimization
Recent research has seen several advances relevant to black-box variational inference (VI), but the current state of automatic posterior inference is unclear.
Multifaceted Uncertainty Estimation for Label-Efficient Deep Learning
DL based evidence prediction with uncertainty anchor sample identification.
Simultaneous Missing Value Imputation and Structure Learning with Groups Pablo Morales-Alvarez University of Granada Wenbo Gong Microsoft Research Angus Lamb
For many applications, variables in the data can be gathered into semantically meaningful groups, where useful insights are at group level. For example, in finance, one may be interested in how a financial situation influences different industries (i.e.
A Additional Background on Bayesian neural networks and variational inference Consider a training set comprising of N input-output pairs, D = { x
Neal, 2012, Blundell et al., 2015], and (iii) using structured variational approximations that can potentially capture weight correlations in the posterior [Louizos and Welling, 2016, Zhang et al., We also vary the amount of inducing points we afford each kernel. The main difference in the local model is the dependence of weights on inputs.
Lower Bounds on Metropolized Sampling Methods for Well-Conditioned Distributions Yin Tat Lee Ruoqi Shen Kevin Tian
Sampling from a continuous distribution in high dimensions is a fundamental problem in algorithm design. As sampling serves as a key subroutine in a variety of tasks in machine learning [AdFDJ03], statistical methods [RC99], and scientific computing [Liu01], it is an important undertaking to understand the complexity of sampling from families of distributions arising in applications. The more restricted problem of sampling from a particular family of distributions, which we call "well-conditioned distributions," has garnered a substantial amount of recent research effort from the algorithmic learning and statistics communities. This specific family is interesting for a number of reasons. First of all, it is practically relevant: Bayesian methods have found increasing use in machine learning applications [Bar12], and many distributions arising from these methods are well-conditioned, such as multivariate Gaussians, mixture models with small separation, and densities arising from Bayesian logistic regression with a Gaussian prior [DCWY18].
An Infinite-Feature Extension for Bayesian ReLU Nets That Fixes Their Asymptotic Overconfidence
While this may seem of mostly theoretical interest, in this work, we show that it can be used in practice to the benefit of BNNs.
Structure learning in polynomial time: Greedy algorithms, Bregman information, and exponential families
In practice, however, they are very efficient.