Goto

Collaborating Authors

 Government


A Skewness-Based Criterion for Addressing Heteroscedastic Noise in Causal Discovery

arXiv.org Machine Learning

Real-world data often violates the equal-variance assumption (homoscedasticity), making it essential to account for heteroscedastic noise in causal discovery. In this work, we explore heteroscedastic symmetric noise models (HSNMs), where the effect $Y$ is modeled as $Y = f(X) + \sigma(X)N$, with $X$ as the cause and $N$ as independent noise following a symmetric distribution. We introduce a novel criterion for identifying HSNMs based on the skewness of the score (i.e., the gradient of the log density) of the data distribution. This criterion establishes a computationally tractable measurement that is zero in the causal direction but nonzero in the anticausal direction, enabling the causal direction discovery. We extend this skewness-based criterion to the multivariate setting and propose SkewScore, an algorithm that handles heteroscedastic noise without requiring the extraction of exogenous noise. We also conduct a case study on the robustness of SkewScore in a bivariate model with a latent confounder, providing theoretical insights into its performance. Empirical studies further validate the effectiveness of the proposed method.


Think While You Generate: Discrete Diffusion with Planned Denoising

arXiv.org Machine Learning

Discrete diffusion has achieved state-of-the-art performance, outperforming or approaching autoregressive models on standard benchmarks. In this work, we introduce Discrete Diffusion with Planned Denoising (DDPD), a novel framework that separates the generation process into two models: a planner and a denoiser. At inference time, the planner selects which positions to denoise next by identifying the most corrupted positions in need of denoising, including both initially corrupted and those requiring additional refinement. This plan-and-denoise approach enables more efficient reconstruction during generation by iteratively identifying and denoising corruptions in the optimal order. DDPD outperforms traditional denoiser-only mask diffusion methods, achieving superior results on language modeling benchmarks such as text8, OpenWebText, and token-based generation on ImageNet $256 \times 256$. Notably, in language modeling, DDPD significantly reduces the performance gap between diffusion-based and autoregressive methods in terms of generative perplexity. Code is available at https://github.com/liusulin/DDPD.


Temperature Optimization for Bayesian Deep Learning

arXiv.org Machine Learning

The Cold Posterior Effect (CPE) is a phenomenon in Bayesian Deep Learning (BDL), where tempering the posterior to a cold temperature often improves the predictive performance of the posterior predictive distribution (PPD). Although the term `CPE' suggests colder temperatures are inherently better, the BDL community increasingly recognizes that this is not always the case. Despite this, there remains no systematic method for finding the optimal temperature beyond grid search. In this work, we propose a data-driven approach to select the temperature that maximizes test log-predictive density, treating the temperature as a model parameter and estimating it directly from the data. We empirically demonstrate that our method performs comparably to grid search, at a fraction of the cost, across both regression and classification tasks. Finally, we highlight the differing perspectives on CPE between the BDL and Generalized Bayes communities: while the former primarily focuses on predictive performance of the PPD, the latter emphasizes calibrated uncertainty and robustness to model misspecification; these distinct objectives lead to different temperature preferences.


DisasterQA: A Benchmark for Assessing the performance of LLMs in Disaster Response

arXiv.org Artificial Intelligence

Disasters can result in the deaths of many, making quick response times vital. Large Language Models (LLMs) have emerged as valuable in the field. LLMs can be used to process vast amounts of textual information quickly providing situational context during a disaster. However, the question remains whether LLMs should be used for advice and decision making in a disaster. To evaluate the capabilities of LLMs in disaster response knowledge, we introduce a benchmark: DisasterQA created from six online sources. The benchmark covers a wide range of disaster response topics. We evaluated five LLMs each with four different prompting methods on our benchmark, measuring both accuracy and confidence levels through Logprobs. The results indicate that LLMs require improvement on disaster response knowledge. We hope that this benchmark pushes forth further development of LLMs in disaster response, ultimately enabling these models to work alongside. emergency managers in disasters.


Cooperative and Asynchronous Transformer-based Mission Planning for Heterogeneous Teams of Mobile Robots

arXiv.org Artificial Intelligence

Coordinating heterogeneous teams of mobile robots for tasks such as search and rescue is highly challenging. This is due to the complexities of perception, decision making and planning in such environments, with agents' non-synchronous operation, constrained communication, and limited computational resources. This paper presents the Cooperative and Asynchronous Transformer-based Mission Planning (CATMiP) framework, which leverages multi-agent reinforcement learning (MARL) to effectively coordinate agents with heterogeneous sensing, motion, and actuation capabilities. The framework introduces a Class-based Macro-Action Decentralized Partially Observable Markov Decision Process (CMD-POMDP) model to handle asynchronous decision-making among different agent classes via macro-actions. It also extends the Multi-Agent Transformer (MAT) architecture to facilitate distributed, ad hoc communication among the agents. CATMiP easily adapts to mission complexities and communication constraints, and scales to varying environment sizes and team compositions. Simulations demonstrate its scalability and ability to achieve cooperative mission objectives with two classes of explorer and rescuer agents, even under severe communication constraints. The code is available at https://github.com/mylad13/CATMiP.


Reviews: A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks

Neural Information Processing Systems

The problem of the study is detecting abnormalities within deep neural networks, to detect out-of-distribution inputs, adversarial inputs, and new classes (for class incremental learning). To achieve this, the authors integrate class-conditional Gaussian distributions with a tied covariance (linear discriminant analysis) at various stages of a target neural network and construct distributions over the valid input (in-liers). They use the Mahalanobis distance measure of the Gaussian distribution as a confidence measure (proportional to the log-likelihood). They further enhance the confidence measure by taking Fast Gradient-Sign Method-style steps in the input space to increase the score. Finally, they combine the scores gathered at different layers of the neural network through a linear combination.


Ukraine strikes oil depot in occupied Crimea

BBC News

Footage circulating on social media appeared to show smoke rising over the Feodosia terminal. Local Russian-installed officials told RIA Novosti that efforts to extinguish the fire were ongoing. Meanwhile, the defence ministry in Moscow said that 12 Ukrainian drones were shot down over the peninsula overnight out of a total of 21 launched by Kyiv. In a statement announcing the attack, Ukraine's general staff said that oil products shipped from the terminal were being used to "meet the needs of the Russian occupation army". The facility was previously hit in a Ukrainian drone strike in March.


Sex, radiation and mummies: How farms are fighting a pesky almond moth without pesticides

Los Angeles Times

In a windowless shack on the far outskirts of Fresno, an ominious red glow illuminates a lab filled with X-ray machines, shelves of glowing boxes, a quietly humming incubator and a miniature wind tunnel. While the scene looks like something straight out of a sci-fi movie, its actually part of an experimental program to prevent a damaging almond pest from successfully mating. With California almond growers reeling from dropping nut prices and rising costs, the pests have only added to their woes. Every year, the navel orangeworm eats through roughly 2% of California's almonds before they can make it to grocery store shelves. Last year, it was almost double that.


Selective Labeling via Error Bound Minimization

Neural Information Processing Systems

In many practical machine learning problems, the acquisition of labeled data is often expensive and/or time consuming. This motivates us to study a problem as follows: given a label budget, how to select data points to label such that the learning performance is optimized. We propose a selective labeling method by analyzing the out-of-sample error of Laplacian regularized Least Squares (LapRLS). In particular, we derive a deterministic out-of-sample error bound for LapRLS trained on subsampled data, and propose to select a subset of data points to label by minimizing this upper bound. Since the minimization is a combinational problem, we relax it into continuous domain and solve it by projected gradient descent. Experiments on benchmark datasets show that the proposed method outperforms the state-of-the-art methods.


Netherlands to provide 400 million to develop drones with Ukraine

The Japan Times

Dutch Defense Minister Ruben Brekelmans said while on a surprise visit to Kyiv on Sunday that his country will invest 400 million ( 440 million) in advanced drone development with Ukraine and deliver more F-16s in the coming months. More than 2½ years since the start of the Russian full-scale invasion, Ukraine is fighting to thwart Russia's troops as they inch forward in the east and attack critical infrastructure ahead of the winter months. "The war, of course, is intensifying every day, and Ukraine is setting up more brigades who all need support, who all need military equipment. We need to have this continuous flow of support," Brekelmans said in Kyiv.