Industry
Amortized Simulation-Based Inference in Generalized Bayes via Neural Posterior Estimation
Sun, Shiyi, Nicholls, Geoff K., Lee, Jeong Eun
Generalized Bayesian Inference (GBI) tempers a loss with a temperature $ฮฒ>0$ to mitigate overconfidence and improve robustness under model misspecification, but existing GBI methods typically rely on costly MCMC or SDE-based samplers and must be re-run for each new dataset and each $ฮฒ$ value. We give the first fully amortized variational approximation to the tempered posterior family $p_ฮฒ(ฮธ\mid x) \propto ฯ(ฮธ)\,p(x \mid ฮธ)^ฮฒ$ by training a single $(x,ฮฒ)$-conditioned neural posterior estimator $q_ฯ(ฮธ\mid x,ฮฒ)$ that enables sampling in a single forward pass, without simulator calls or inference-time MCMC. We introduce two complementary training routes: (i) synthesize off-manifold samples $(ฮธ,x) \sim ฯ(ฮธ)\,p(x \mid ฮธ)^ฮฒ$ and (ii) reweight a fixed base dataset $ฯ(ฮธ)\,p(x \mid ฮธ)$ using self-normalized importance sampling (SNIS). We show that the SNIS-weighted objective provides a consistent forward-KL fit to the tempered posterior with finite weight variance. Across four standard simulation-based inference (SBI) benchmarks, including the chaotic Lorenz-96 system, our $ฮฒ$-amortized estimator achieves competitive posterior approximations in standard two-sample metrics, matching non-amortized MCMC-based power-posterior samplers over a wide range of temperatures.
Neural-Inspired Posterior Approximation (NIPA)
Shahbaba, Babak, Moslemi, Zahra
Humans learn efficiently from their environment by engaging multiple interacting neural systems that support distinct yet complementary forms of control, including model-based (goal-directed) planning, model-free (habitual) responding, and episodic memory-based learning. Model-based mechanisms compute prospective action values using an internal model of the environment, supporting flexible but computationally costly planning; model-free mechanisms cache value estimates and build heuristics that enable fast, efficient habitual responding; and memory-based mechanisms allow rapid adaptation from individual experience. In this work, we aim to elucidate the computational principles underlying this biological efficiency and translate them into a sampling algorithm for scalable Bayesian inference through effective exploration of the posterior distribution. More specifically, our proposed algorithm comprises three components: a model-based module that uses the target distribution for guided but computationally slow sampling; a model-free module that uses previous samples to learn patterns in the parameter space, enabling fast, reflexive sampling without directly evaluating the expensive target distribution; and an episodic-control module that supports rapid sampling by recalling specific past events (i.e., samples). We show that this approach advances Bayesian methods and facilitates their application to large-scale statistical machine learning problems. In particular, we apply our proposed framework to Bayesian deep learning, with an emphasis on proper and principled uncertainty quantification.
RPWithPrior: Label Differential Privacy in Regression
With the wide application of machine learning techniques in practice, privacy preservation has gained increasing attention. Protecting user privacy with minimal accuracy loss is a fundamental task in the data analysis and mining community. In this paper, we focus on regression tasks under $ฮต$-label differential privacy guarantees. Some existing methods for regression with $ฮต$-label differential privacy, such as the RR-On-Bins mechanism, discretized the output space into finite bins and then applied RR algorithm. To efficiently determine these finite bins, the authors rounded the original responses down to integer values. However, such operations does not align well with real-world scenarios. To overcome these limitations, we model both original and randomized responses as continuous random variables, avoiding discretization entirely. Our novel approach estimates an optimal interval for randomized responses and introduces new algorithms designed for scenarios where a prior is either known or unknown. Additionally, we prove that our algorithm, RPWithPrior, guarantees $ฮต$-label differential privacy. Numerical results demonstrate that our approach gets better performance compared with the Gaussian, Laplace, Staircase, and RRonBins, Unbiased mechanisms on the Communities and Crime, Criteo Sponsored Search Conversion Log, California Housing datasets.
Approximating $f$-Divergences with Rank Statistics
Stein, Viktor, de Frutos, Josรฉ Manuel
We introduce a rank-statistic approximation of $f$-divergences that avoids explicit density-ratio estimation by working directly with the distribution of ranks. For a resolution parameter $K$, we map the mismatch between two univariate distributions $ฮผ$ and $ฮฝ$ to a rank histogram on $\{ 0, \ldots, K\}$ and measure its deviation from uniformity via a discrete $f$-divergence, yielding a rank-statistic divergence estimator. We prove that the resulting estimator of the divergence is monotone in $K$, is always a lower bound of the true $f$-divergence, and we establish quantitative convergence rates for $K\to\infty$ under mild regularity of the quantile-domain density ratio. To handle high-dimensional data, we define the sliced rank-statistic $f$-divergence by averaging the univariate construction over random projections, and we provide convergence results for the sliced limit as well. We also derive finite-sample deviation bounds along with asymptotic normality results for the estimator. Finally, we empirically validate the approach by benchmarking against neural baselines and illustrating its use as a learning objective in generative modelling experiments.
GRANITE: A Generalized Regional Framework for Identifying Agreement in Feature-Based Explanations
Herbinger, Julia, Laberge, Gabriel, Muschalik, Maximilian, Pequignot, Yann, Wright, Marvin N., Fumagalli, Fabian
Feature-based explanation methods aim to quantify how features influence the model's behavior, either locally or globally, but different methods often disagree, producing conflicting explanations. This disagreement arises primarily from two sources: how feature interactions are handled and how feature dependencies are incorporated. We propose GRANITE, a generalized regional explanation framework that partitions the feature space into regions where interaction and distribution influences are minimized. This approach aligns different explanation methods, yielding more consistent and interpretable explanations. GRANITE unifies existing regional approaches, extends them to feature groups, and introduces a recursive partitioning algorithm to estimate such regions. We demonstrate its effectiveness on real-world datasets, providing a practical tool for consistent and interpretable feature explanations.
A Random Matrix Theory of Masked Self-Supervised Regression
Zurich, Arie Wortsman, Gerace, Federica, Loureiro, Bruno, Lu, Yue M.
Self-supervised learning (SSL) -- a training paradigm in which models learn useful representations from unlabeled data by exploiting the data itself as a source of supervision -- has emerged as a foundational component of the recent success of transformer architectures. By avoiding the need for manual annotations, SSL retains many of the benefits traditionally associated with supervised learning while avoiding reliance on labeled data. Consequently, SSL is widely adopted as a pretraining paradigm for learning general-purpose representations that substantially accelerate the optimization of downstream tasks, especially in data-scarce settings. A canonical example of a self-supervised learning task is masked language modeling (MLM), in which a neural network is trained to predict masked tokens in text using the remaining tokens as contextual information (Devlin et al., 2019a; Howard and Ruder, 2018; Radford et al., 2018; Brown et al., 2020; OpenAI, 2024). For example, given the sentence "The capital of France is Paris", a typical MLM task would be to teach the model to infer that we are speaking about the capital of a country from the context "France" and "Paris" from the masked sentence "The [MASK] of France is Paris".
Asymptotic Theory of Iterated Empirical Risk Minimization, with Applications to Active Learning
We study a class of iterated empirical risk minimization (ERM) procedures in which two successive ERMs are performed on the same dataset, and the predictions of the first estimator enter as an argument in the loss function of the second. This setting, which arises naturally in active learning and reweighting schemes, introduces intricate statistical dependencies across samples and fundamentally distinguishes the problem from classical single-stage ERM analyses. For linear models trained with a broad class of convex losses on Gaussian mixture data, we derive a sharp asymptotic characterization of the test error in the high-dimensional regime where the sample size and ambient dimension scale proportionally. Our results provide explicit, fully asymptotic predictions for the performance of the second-stage estimator despite the reuse of data and the presence of prediction-dependent losses. We apply this theory to revisit a well-studied pool-based active learning problem, removing oracle and sample-splitting assumptions made in prior work. We uncover a fundamental tradeoff in how the labeling budget should be allocated across stages, and demonstrate a double-descent behavior of the test error driven purely by data selection, rather than model size or sample count.
Twelve miners killed by Russian strike in Ukraine, energy company says
Twelve miners have been killed by a Russian drone strike in eastern Ukraine, the country's largest private energy firm has said. DTEK said a bus carrying workers after a shift in the Dnipropetrovsk region had been targeted in Sunday's attack. At least seven people were injured. Earlier, at least two others were killed and nine injured in separate Russian attacks overnight and on Sunday. The victims included six people hurt when a drone hit a maternity hospital in Zaporizhzhia.
Russian drone attack on bus in Ukraine kills at least 12
Could Ukraine hold a presidential election right now? Will Europe use frozen Russian assets to fund war? How can Ukraine rebuild China ties? 'Ukraine is running out of men, money and time' At least 12 people have been killed in a Russian drone attack on a bus carrying miners in Ukraine's southeastern Dnipropetrovsk region, the country's energy minister said. "Today, the enemy carried out a cynical and targeted attack on energy sector workers in the Dnipro region," Minister of Energy Denys Shmyhal posted on Telegram on Sunday.
Indonesia is lifting its ban on Grok, but with some conditions
The country's Ministry of Communication and Digital Affairs said it will monitor xAI's newly implemented safety measures on an ongoing basis. Grok is once again available in Indonesia, after the country lifted its ban on the AI chatbot that was seen generating millions of sexualized deepfakes, thousands of which included children. The country's Ministry of Communication and Digital Affairs released a statement earlier today, which said X is allowed to resume service in Indonesia but will be subject to monitoring for any future violations. According to the Indonesian government agency, X provided a letter that detailed several implemented measures that prevent the misuse of its Grok chatbot. Alexander Sabar, the ministry's director general of digital space supervision, said in the statement that the agency will test the new measures on an ongoing basis and will ban Grok again if it's found spreading illegal content or violating the country's laws regarding children.