Bayesian Learning
A Game Theoretic Analysis of Additive Adversarial Attacks and Defenses
In this paper, we propose a game-theoretic framework for studying attacks and defenses which exist in equilibrium.
Deep Gamblers: Learning to Abstain with Portfolio Theory
Ziyin Liu, Zhikang Wang, Paul Pu Liang, Russ R. Salakhutdinov, Louis-Philippe Morency, Masahito Ueda
Top-10 rejected images in the MNIST testing set found by two methods.
Fast Classification Rates for High-dimensional Gaussian Generative Models
Tianyang Li, Adarsh Prasad, Pradeep K. Ravikumar
Neural Information Processing Systems http://nips.cc/
use [Narasimhan et al. '15 ] for Narasimhan Het al., Learnability of influence in networks, NeurIPS'2015. 2 - Reviewer 3: About the setting of online linear threshold model
We thank the reviewers for the valuable comments and discussions. Please find our clarifications below. IC model [11,43,45], which also learns unknown edge probability parameters. It is interesting that the reviewer brought up the frequentist versus Bayesian view on OIM-LT. LT model and our work is a frequentist approach for the online setting.
Sampling from Probabilistic Submodular Models
Alkis Gotovos, Hamed Hassani, Andreas Krause
Submodular and supermodular functions have found wide applicability in machine learning, capturing notions such as diversity and regularity, respectively. These notions have deep consequences for optimization, and the problem of (approximately) optimizing submodular functions has received much attention. However, beyond optimization, these notions allow specifying expressive probabilistic models that can be used to quantify predictive uncertainty via marginal inference. Prominent, well-studied special cases include Ising models and determinan-tal point processes, but the general class of log-submodular and log-supermodular models is much richer and little studied. In this paper, we investigate the use of Markov chain Monte Carlo sampling to perform approximate inference in general log-submodular and log-supermodular models. In particular, we consider a simple Gibbs sampling procedure, and establish two sufficient conditions, the first guaranteeing polynomial-time, and the second fast ( O ( n log n)) mixing. We also evaluate the efficiency of the Gibbs sampler on three examples of such models, and compare against a recently proposed variational approach.
Maximum Likelihood Learning With Arbitrary Treewidth via Fast-Mixing Parameter Sets
Neural Information Processing Systems http://nips.cc/
Reliable training and estimation of variance networks
Nicki Skafte, Martin Jørgensen, Søren Hauberg
We propose and investigate new complementary methodologies for estimating predictive variance networks in regression neural networks.
Statistical Model Criticism using Kernel Two Sample Tests
James R. Lloyd, Zoubin Ghahramani
Neural Information Processing Systems http://nips.cc/
Modeling Uncertainty by Learning a Hierarchy of Deep Neural Connections
Raanan Yehezkel Rohekar, Yaniv Gurwicz, Shami Nisimov, Gal Novik
Neural Information Processing Systems http://nips.cc/
How Does the Pretraining Distribution Shape In-Context Learning? Task Selection, Generalization, and Robustness
The emergence of in-context learning (ICL) in large language models (LLMs) remains poorly understood despite its consistent effectiveness, enabling models to adapt to new tasks from only a handful of examples. To clarify and improve these capabilities, we characterize how the statistical properties of the pretraining distribution (e.g., tail behavior, coverage) shape ICL on numerical tasks. We develop a theoretical framework that unifies task selection and generalization, extending and sharpening earlier results, and show how distributional properties govern sample efficiency, task retrieval, and robustness. To this end, we generalize Bayesian posterior consistency and concentration results to heavy-tailed priors and dependent sequences, better reflecting the structure of LLM pretraining data. We then empirically study how ICL performance varies with the pretraining distribution on challenging tasks such as stochastic differential equations and stochastic processes with memory. Together, these findings suggest that controlling key statistical properties of the pretraining distribution is essential for building ICL-capable and reliable LLMs.