Bayesian Inference
Ranking with Confidence for Large Scale Comparison Data
Valdeira, Filipa, Soares, Clรกudia
In this work, we leverage a generative data model considering comparison noise to develop a fast, precise, and informative ranking algorithm from pairwise comparisons that produces a measure of confidence on each comparison. The problem of ranking a large number of items from noisy and sparse pairwise comparison data arises in diverse applications, like ranking players in online games, document retrieval or ranking human perceptions. Although different algorithms are available, we need fast, large-scale algorithms whose accuracy degrades gracefully when the number of comparisons is too small. Fitting our proposed model entails solving a non-convex optimization problem, which we tightly approximate by a sum of quasi-convex functions and a regularization term. Resorting to an iterative reweighted minimization and the Primal-Dual Hybrid Gradient method, we obtain PD-Rank, achieving a Kendall tau 0.1 higher than all comparing methods, even for 10\% of wrong comparisons in simulated data matching our data model, and leading in accuracy if data is generated according to the Bradley-Terry model, in both cases faster by one order of magnitude, in seconds. In real data, PD-Rank requires less computational time to achieve the same Kendall tau than active learning methods.
A note on the relations between mixture models, maximum-likelihood and entropic optimal transport
Vayer, Titouan, Lasalle, Etienne
The relations between maximum-likelihood and optimal transport (OT) have already been discussed in multiple works (Rigollet and Weed, 2018; Mena et al., 2020; Diebold et al., 2024). The purpose of this brief note is to provide the key tools used to establish these connections. The primary aim is pedagogical: we will focus on the (discrete) mixtures case, adopting a "computational OT" perspective. Hopefully, readers will find this exercise insightful. Our analysis will largely rely on the approach described in Rigollet and Weed (2018), though adapted to a different formalism and applied to a slightly different problem (mixture estimation rather than Gaussian deconvolution).
Reviews: Poisson-Minibatching for Gibbs Sampling with Convergence Rate Guarantees
Summary: This paper introduces Poisson auxiliary variables to facilitate minibatch sampling. The key insight is with the appropriate Poisson parameterization, the joint distribution (Eq. The authors apply this insight to discrete-state Gibbs sampling (Algorithm 2), Metropolis Hastings (Supplement), and continuous-state Gibbs sampling (Alg 3. and 5). The authors also develop spectral gap lower bounds for all proposed Gibbs sampling methods, which provides a rough guideline for choosing a tuning parameter \lambda and comparing the (asymptotic) per iteration runtime of the methods (Table 1). Finally the authors evaluate the Gibbs methods on synthetic data, showing that their proposed method performs similarly to Gibbs while outperforming alternatives.
Reviews: Debiased Bayesian inference for average treatment effects
In particular, I welcome the comparison to BCF and find it quite interesting that it performs better on the semi-synthetic data but not the synthetic! On re-reading the paper and supplement, as well as the response, I do think the way you have incorporated the propensity score is quite clever. I'm quite happy to revise my score up to 7. ----- The authors consider the important problem of heterogeneous treatment effect estimation. They specifically propose a non-parametric Bayesian procedure, placing a Gaussian process prior on the potential outcome function m(x,r). They note that the natural approach (i.e.
Reviews: Debiased Bayesian inference for average treatment effects
Overall, the reviewers found this a valuable addition to the causal inference literature. While we would have liked to see more comparisons, we feel that that by incorporating the BCF simulations, and the clarifications mentioned in the rebuttal, this paper will be a welcome addition to the conference.
Review for NeurIPS paper: Bidirectional Convolutional Poisson Gamma Dynamical Systems
Summary and Contributions: The paper presents a new hierarchical Bayesian model -- convolutional Poisson-Gamma Dynamical Systems (conv-PGDS) -- for generating the observed words in a document corpus. Globally, the model assumes there are K "topic filters", D_1, ... D_K, which are distributions over 3-grams from a finite size vocabulary (size V). Each "topic" (indexed by k) has an appearance probability weight v_k 0 for appearing in a document, and we define transition probability vectors \pi_k Given this global structure, the model generates each document iid. To generate a document j, we use a Gamma dynamical system (with transitions \pi) to obtain a sequence of un-normalized membership "weight embeddings", w_j1 ... w_jT, one for each sentence (indexed by t). Each weight embedding vector w_jt indicates the relative weight of topic k across all words in the sentence t.
Reviews: An Adaptive Empirical Bayesian Method for Sparse Deep Learning
This is a novel combination of existing techniques that appears well-formulated with intriguing experimental results. In particular, this work leverages the strengths stochastic gradient MCMC methods with stochastic approximation to form an adaptive empirical Bayesian approach to learning the parameters and hyperparameters of a Bayesian neural network (BNN). My best understanding is that by optimizing the hyperparameters (rather than sampling them), this new method improves upon existing approaches, speeding up inference without sacrificing quality (especially in the model compression domain). Other areas of BNN literature could be cited, but I think the authors were prudent not to distract the reader from the particular area of focus. This work demonstrates considerable theoretical analysis and is supported by intriguing experimental evidence.
Attention-Driven Hierarchical Reinforcement Learning with Particle Filtering for Source Localization in Dynamic Fields
Shi, Yiwei, Yang, Mengyue, Zhang, Qi, Zhang, Weinan, Liu, Cunjia, Liu, Weiru
In many real-world scenarios, such as gas leak detection or environmental pollutant tracking, solving the Inverse Source Localization and Characterization problem involves navigating complex, dynamic fields with sparse and noisy observations. Traditional methods face significant challenges, including partial observability, temporal and spatial dynamics, out-of-distribution generalization, and reward sparsity. To address these issues, we propose a hierarchical framework that integrates Bayesian inference and reinforcement learning. The framework leverages an attention-enhanced particle filtering mechanism for efficient and accurate belief updates, and incorporates two complementary execution strategies: Attention Particle Filtering Planning and Attention Particle Filtering Reinforcement Learning. These approaches optimize exploration and adaptation under uncertainty. Theoretical analysis proves the convergence of the attention-enhanced particle filter, while extensive experiments across diverse scenarios validate the framework's superior accuracy, adaptability, and computational efficiency. Our results highlight the framework's potential for broad applications in dynamic field estimation tasks.