Goto

Collaborating Authors

 Genre


Optimizing Computational-Statistical Runtime for Wasserstein Distance Estimation

arXiv.org Machine Learning

Squared Wasserstein distance is a frequently used tool to measure discrepancy between probability distributions. This distance is typically computed between empirical measures of size $n$ from two underlying random samples. Unfortunately, even in lower dimensional Euclidean space problems $\left( d \in \{2,3\} \right)$, algorithms for Wasserstein distance computation with approximate or exact precision guarantees scale poorly in the runtime as a function of $n$ and the desired precision. In response, we consider the computational-statistical runtime, where the goal is to estimate from samples the Wasserstein distance between potentially smooth measures up to $ฮต$-additive error in expectation with respect to the sampling; we allow $O(1)$ computational cost for collecting a sample. Towards this, we develop a Sample-Sketch-Solve paradigm where we introduce a regular cartesian grid sketch of the samples. We show that (especially under $ฮฑ$-Hรถlder smooth distributions) this can compress the data without increasing asymptotic error, and also regularizes the structure which enables faster exact algorithms. Ultimately, we approximate $W_2^2(P,Q)$ within $ฮต$ error in $ฮต^{-\max(2,\frac{d+1+o(1)}{1+ฮฑ})}$ time for $0 < ฮฑ< 1$ Hรถlder smooth distributions $P,Q$ on $(0,1)^{d}$; an optimal $ฮ˜(ฮต^{-2})$ for $ฮฑ> 1/2$ when $d=2$ and nearly optimal as $ฮฑ\to 1$ when $d = 3$.


Goal-Oriented Lower-Tail Calibration of Gaussian Processes for Bayesian Optimization

arXiv.org Machine Learning

Bayesian optimization (BO) selects evaluation points for expensive black-box objectives using Gaussian process (GP) predictive distributions. Kernel choice and hyperparameter selection can lead to miscalibrated predictive distributions and an inappropriate exploration-exploitation trade-off. For minimization, sampling criteria such as expected improvement (EI) depend on the predictive distribution below the current best value, so lower-tail miscalibration directly affects the sampling decision. This article studies goal-oriented calibration of GP predictive distributions below a low threshold $t$ in the noiseless setting, for standard GP models with hyperparameters selected by maximum likelihood. A framework for predictive reliability below $t$ is introduced, based on two notions of spatial calibration: occurrence calibration over the design space and thresholded $ฮผ$-calibration on sublevel sets of the form $\{x\in\mathbb{X}, f(x)\le t\}$. Building on this framework, we propose tcGP, a post-hoc method that calibrates GP predictive distributions below~$t$, and we show that the resulting EI-based global optimization algorithm remains dense in the design space. Experiments on standard benchmarks show improved lower-tail calibration and BO performance relative to standard GP models and globally calibrated GP models.


'Obvious markers of AI': doubts raised over winner of short story prize

The Guardian

The Commonwealth Foundation said all entrants to the prize had avowed that their submissions were their own work. The Commonwealth Foundation said all entrants to the prize had avowed that their submissions were their own work. 'Obvious markers of AI': doubts raised over winner of short story prize Granta publisher says'perhaps we never will know' true authorship of work that won Commonwealth prize A few syntactical tics - and the verdict of an AI detection platform - have sparked a furore over the possibility that a short story given a prestigious literary award was written by AI. The foundation that awarded the prize and Granta, the magazine that published the winning story, said they had considered the allegations but had not reached a conclusion as to whether they were true. "It may be that the judges have now awarded a prize to an instance of AI plagiarism - we don't yet know, and perhaps we never will know," the publisher of Granta, Sigrid Rausing, said.


Newly discovered spider has smiley face on its back

Popular Science

'I knew instantly we had a jackpot.' More information Adding us as a Preferred Source in Google by using this link indicates that you would like to see more of our content in Google News results. Both species appear to have a preference for ginger plants. Breakthroughs, discoveries, and DIY tips sent six days a week. The happy-face spider () is famous for the particularly cheery looking patterns on top of its abdomen.


Worker bees have power to pick their queen

Popular Science

Bumble bees either become the larger and fertile queen or a smaller and sterile worker. More information Adding us as a Preferred Source in Google by using this link indicates that you would like to see more of our content in Google News results. Worker bees can alter the future of the colony's larvae. Breakthroughs, discoveries, and DIY tips sent six days a week. While every bumble bee colony has a queen, the process for becoming that queen bee may be a bit more democratic than monarchical.


The Zuckerbergs Are Hiring a Lifeguard but Calling It a 'Beach Water Person'

WIRED

The Zuckerbergs Are Hiring a Lifeguard but Calling It a'Beach Water Person' The job, which is associated with the Zuckerberg family office, is located in Kauai, Hawaii, where the Meta CEO owns a massive compound. Meta CEO Mark Zuckerberg and his wife Priscilla Chan are hiring a seasonal, on-call "Beach Water Person" based in Kauai, Hawaii, where the family owns a sprawling compound, according to a new job listing on Greenhouse associated with West 10, the Zuckerberg family office. This is an interesting choice for a job title, because according to the job description, the primary duties of this "Beach Water Person" include serving as a "Beach Lifeguard," and "Pool Lifeguard." The job listing names a few additional duties related to water activities, such as instructing "stand-up paddleboarding (SUP), canoe paddling, snorkeling, and other ocean-based activities." These, however, come after the water safety duties in the job description.


Kernelized Advantage Estimation: From Nonparametric Statistics to LLM Reasoning

arXiv.org Machine Learning

Recent advances in large language models (LLMs) have increasingly relied on reinforcement learning (RL) to improve their reasoning capabilities. Three types of approaches have been widely adopted: The first relies on a deep neural network to estimate the value function of the learning policy in order to reduce the variance of the policy gradient. However, estimating and maintaining such a value network incurs substantial computational and memory overhead. The second avoids training a value network by approximating the value function using sample averages. However, it samples a large number of reasoning traces per prompt for accurate value function approximation, making it computationally expensive. The third samples only a single reasoning trajectory per prompt, which reduces computational cost but suffers from poor sample efficiency. This paper focuses on a practical, resource-constrained setting in which only a small number of reasoning traces can be sampled per prompt, while low-variance gradient estimation remains essential for high-quality policy learning. To address this challenge, we bring classical nonparametric statistical methods, which are both computationally and statistically efficient, to LLM reasoning. We employ kernel smoothing as a concrete example for value function estimation and the subsequent policy optimization. Numerical and theoretical results demonstrate that our proposal achieves accurate value and gradient estimation, leading to improved policy optimization.


Wasserstein Distributionally Robust Regret Optimization for Reinforcement Learning from Human Feedback

arXiv.org Machine Learning

Reinforcement learning from human feedback (RLHF) has become a core post-training step for aligning large language models, yet the reward signal used in RLHF is only a learned proxy for true human utility. From an operations research perspective, this creates a decision problem under objective misspecification: the policy is optimized against an estimated reward, while deployment performance is determined by an unobserved objective. The resulting gap leads to reward over-optimization, or Goodharting, where proxy reward continues to improve even after true quality deteriorates. Existing mitigations address this problem through uncertainty penalties, pessimistic rewards, or conservative constraints, but they can be computationally burdensome and overly pessimistic. We propose Wasserstein distributionally robust regret optimization (DRRO) for RLHF. Instead of pessimizing worst-case value as in standard DRO, DRRO pessimizes worst-case regret relative to the best policy under the same plausible reward perturbation. We study the promptwise problem through a simplex allocation model and show that, under an $\ell_1$-ground-cost Wasserstein ambiguity set, the inner worst-case regret admits an exact solution and the optimal policy has a water-filling structure. These results lead to a practical policy-gradient algorithm with a simple sampled-bonus interpretation and only minor changes to GRPO-style RLHF training. The framework also clarifies theoretically why DRRO is less pessimistic than DRO, and our experiments show that DRRO mitigates over-optimization more effectively than existing baselines while standard DRO is systematically over-pessimistic.


Understanding Self-Supervised Learning via Latent Distribution Matching

arXiv.org Machine Learning

Self-supervised learning (SSL) excels at finding general-purpose latent representations from complex data, yet lacks a unifying theoretical framework that explains the diverse existing methods and guides the design of new ones. We cast SSL as latent distribution matching (LDM): learning representations that maximize their log-probability under an assumed latent model (alignment), while maximizing latent entropy to prevent collapse (uniformity). This view unifies independent component analysis with contrastive, non-contrastive, and predictive SSL methods, including stop gradient approaches. Leveraging LDM, we derive a nonlinear, sampling-free Bayesian filtering model with a Kalman-based predictor for high-dimensional timeseries. We further prove that predictive LDM yields identifiable latent representations under mild assumptions, even with nonlinear predictors. Overall, LDM clarifies the assumptions behind established SSL methods and provides principled guidance for developing new approaches.


Resource-Element Energy Difference for Noncoherent Over-the-Air Federated Learning

arXiv.org Machine Learning

Over-the-air federated learning (OTA-FL) reduces uplink latency by aggregating client updates directly over the wireless multiple-access channel. Coherent analog aggregation realizes this idea by aligning the phases and amplitudes of simultaneously transmitted waveforms, which typically requires synchronization, instantaneous channel-state information (CSI), phase compensation, and power control. Noncoherent energy detection removes the need for phase-coherent combining, but a single energy measurement is nonnegative and, therefore, cannot represent signed model updates. This paper introduces resource-element energy difference (REED), a noncoherent physical-layer primitive for continuous signed aggregation. REED maps the positive and negative parts of each real-valued update to transmit energies on paired orthogonal resource elements and estimates the signed sum by subtracting the corresponding received energies. The construction uses slow-timescale calibration of average channel powers, but does not require instantaneous transmitter- or receiver-side CSI or channel inversion. For independent Rayleigh fading, we derive exact first- and second-moment expressions for single-shot REED and for a chip-diverse extension that spreads each coordinate over multiple independently faded paired chips. The resulting variance laws separate fading-induced self-noise, signal-noise interaction, and receiver-noise fluctuation, giving an explicit diversity-resource tradeoff. More->The rest of abstract is in the paper.