Goto

Collaborating Authors

 Deep Learning


FairGRPO: Fair Reinforcement Learning for Equitable Clinical Reasoning

arXiv.org Artificial Intelligence

Medical artificial intelligence systems have achieved remarkable diagnostic capabilities, yet they consistently exhibit performance disparities across demographic groups, causing real-world harm to underrepresented populations. While recent multimodal reasoning foundation models have advanced clinical diagnosis through integrated analysis of diverse medical data, reasoning trainings via reinforcement learning inherit and often amplify biases present in training datasets dominated by majority populations. We introduce Fairness-aware Group Relative Policy Optimization (FairGRPO), a hierarchical reinforcement learning approach that promotes equitable learning across heterogeneous clinical populations. FairGRPO employs adaptive importance weighting of advantages based on representation, task difficulty, and data source. To address the common issue of missing demographic labels in the clinical domain, we further employ unsupervised clustering, which automatically discovers latent demographic groups when labels are unavailable. Through comprehensive experiments across 7 clinical diagnostic datasets spanning 5 clinical modalities across X-ray, CT scan, dermoscropy, mammography and ultrasound, we demonstrate that FairGRPO reduces predictive parity by 27.2% against all vanilla and bias mitigated RL baselines, while improving F1 score by 12.49%. Furthermore, training dynamics analysis reveals that FairGRPO progressively improves fairness throughout optimization, while baseline RL methods exhibit deteriorating fairness as training progresses. Based on FairGRPO, we release FairMedGemma-4B, a fairness-aware clinical VLLM that achieves state-of-the-art performance while demonstrating significantly reduced disparities across demographic groups.


Can They Dixit? Yes they Can! Dixit as a Playground for Multimodal Language Model Capabilities

arXiv.org Artificial Intelligence

Multi-modal large language models (MLMs) are often assessed on static, individual benchmarks -- which cannot jointly assess MLM capabilities in a single task -- or rely on human or model pairwise comparisons -- which is highly subjective, expensive, and allows models to exploit superficial shortcuts (e.g., verbosity) to inflate their win-rates. To overcome these issues, we propose game-based evaluations to holistically assess MLM capabilities. Games require multiple abilities for players to win, are inherently competitive, and are governed by fix, objective rules, and makes evaluation more engaging, providing a robust framework to address the aforementioned challenges. We manifest this evaluation specifically through Dixit, a fantasy card game where players must generate captions for a card that trick some, but not all players, into selecting the played card. Our quantitative experiments with five MLMs show Dixit win-rate rankings are perfectly correlated with those on popular MLM benchmarks, while games between human and MLM players in Dixit reveal several differences between agent strategies and areas of improvement for MLM reasoning.


Deep Sequence-to-Sequence Models for GNSS Spoofing Detection

arXiv.org Artificial Intelligence

Abstract--We present a data generation framework designed to simulate spoofing attacks and randomly place attack scenarios worldwide. We apply deep neural network-based models for spoofing detection, utilizing Long Short-T erm Memory networks and Transformer-inspired architectures. These models are specifically designed for online detection and are trained using the generated dataset. Our results demonstrate that deep learning models can accurately distinguish spoofed signals from genuine ones, achieving high detection performance. The best results are achieved by Transformer-inspired architectures with early fusion of the inputs resulting in an error rate of 0.16%. Unencrypted civilian global navigation satellite system (GNSS) signals are vulnerable to spoofing attacks, which pose a significant threat.


From Optimization to Prediction: Transformer-Based Path-Flow Estimation to the Traffic Assignment Problem

arXiv.org Artificial Intelligence

The traffic assignment problem is essential for traffic flow analysis, traditionally solved using mathematical programs under the Equilibrium principle. These methods become computationally prohibitive for large-scale networks due to non-linear growth in complexity with the number of OD pairs. This study introduces a novel data-driven approach using deep neural networks, specifically leveraging the Transformer architecture, to predict equilibrium path flows directly. By focusing on path-level traffic distribution, the proposed model captures intricate correlations between OD pairs, offering a more detailed and flexible analysis compared to traditional link-level approaches. The Transformer-based model drastically reduces computation time, while adapting to changes in demand and network structure without the need for recalculation. Numerical experiments are conducted on the Manhattan-like synthetic network, the Sioux Falls network, and the Eastern-Massachusetts network. The results demonstrate that the proposed model is orders of magnitude faster than conventional optimization. It efficiently estimates path-level traffic flows in multi-class networks, reducing computational costs and improving prediction accuracy by capturing detailed trip and flow information. Introduction The Traffic Assignment Problem (TAP) is a process of determining the propagation of flows over the transportation network. The goal is to calculate the network state, given the travel demand between various origin-destination (OD) pairs and the network's capacity constraints (Y osef Sheffi. Traditionally, this problem is solved through mathematical programs under the User Equilibrium (UE) principle, which assumes drivers possess perfect information and make fully rational choices (Wardrop, 1952). Despite potential deviations from reality, this approach consistently provides reasonable solutions to the traffic assignment problem (Bar-Gera, 2002; Jafari et al., 2017). However, the computation for determining optimal solutions in large traffic networks is prohibitively costly. This is because the problem's complexity grows non-linearly with the increase in the number of OD pairs and directly depends on feasible paths. When the size of the network (the number of links and nodes in a representative graph) increases, allowing us to explore more paths, the number of feasible paths also increases, and the OD demand matrix may grow accordingly, leading to a non-linear increase in computation time (Patriksson, 2015).


Compressing Biology: Evaluating the Stable Diffusion VAE for Phenotypic Drug Discovery

arXiv.org Artificial Intelligence

High-throughput phenotypic screens generate vast microscopy image datasets that push the limits of generative models due to their large dimensionality. Despite the growing popularity of general-purpose models trained on natural images for microscopy data analysis, their suitability in this domain has not been quantitatively demonstrated. We present the first systematic evaluation of Stable Diffusion's variational autoencoder (SD-VAE) for reconstructing Cell Painting images, assessing performance across a large dataset with diverse molecular perturbations and cell types. We find that SD-VAE reconstructions preserve phenotypic signals with minimal loss, supporting its use in microscopy workflows. To benchmark reconstruction quality, we compare pixel-level, embedding-based, latent-space, and retrieval-based metrics for a biologically informed evaluation. We show that general-purpose feature extractors like InceptionV3 match or surpass publicly available bespoke models in retrieval tasks, simplifying future pipelines. Our findings offer practical guidelines for evaluating generative models on microscopy data and support the use of off-the-shelf models in phenotypic drug discovery.


An Expert-grounded benchmark of General Purpose LLMs in LCA

arXiv.org Artificial Intelligence

Purpose: Artificial intelligence (AI), and in particular large language models (LLMs), are increasingly being explored as tools to support life cycle assessment (LCA). While demonstrations exist across environmental and social domains, systematic evidence on their reliability, robustness, and usability remains limited. This study provides the first expert-grounded benchmark of LLMs in LCA, addressing the absence of standardized evaluation frameworks in a field where no clear ground truth or consensus protocols exist. Methods: We evaluated eleven general-purpose LLMs, spanning both commercial and open-source families, across 22 LCA-related tasks. Seventeen experienced practitioners reviewed model outputs against criteria directly relevant to LCA practice, including scientific accuracy, explanation quality, robustness, verifiability, and adherence to instructions. We collected 168 expert reviews. Results: Experts judged 37% of responses to contain inaccurate or misleading information. Ratings of accuracy and quality of explanation were generally rated average or good on many models even smaller models, and format adherence was generally rated favourably. Hallucination rates varied significantly, with some models producing hallucinated citations at rates of up to 40%. There was no clear-cut distinction between ratings on open-weight versus closed-weight LLMs, with open-weight models outperforming or competing on par with closed-weight models on criteria such as accuracy and quality of explanation. Conclusion: These findings highlight the risks of applying LLMs naรฏvely in LCA, such as when LLMs are treated as free-form oracles, while also showing benefits especially around quality of explanation and alleviating labour intensiveness of simple tasks. The use of general-purpose LLMs without grounding mechanisms presents ...


Stream: Scaling up Mechanistic Interpretability to Long Context in LLMs via Sparse Attention

arXiv.org Artificial Intelligence

As Large Language Models (LLMs) scale to million-token contexts, traditional Mechanistic Interpretability techniques for analyzing attention scale quadratically with context length, demanding terabytes of memory beyond 100,000 tokens. We introduce Sparse Tracing, a novel technique that leverages dynamic sparse attention to efficiently analyze long context attention patterns. We present Stream, a compilable hierarchical pruning algorithm that estimates per-head sparse attention masks in near-linear time $O(T \log T)$ and linear space $O(T)$, enabling one-pass interpretability at scale. Stream performs a binary-search-style refinement to retain only the top-$k$ key blocks per query while preserving the model's next-token behavior. We apply Stream to long chain-of-thought reasoning traces and identify thought anchors while pruning 97-99\% of token interactions. On the RULER benchmark, Stream preserves critical retrieval paths while discarding 90-96\% of interactions and exposes layer-wise routes from the needle to output. Our method offers a practical drop-in tool for analyzing attention patterns and tracing information flow without terabytes of caches. By making long context interpretability feasible on consumer GPUs, Sparse Tracing helps democratize chain-of-thought monitoring. Code is available at https://anonymous.4open.science/r/stream-03B8/.


From Large to Small: Transferring CUDA Optimization Expertise via Reasoning Graph

arXiv.org Artificial Intelligence

Despite significant evolution of CUDA programming and domain-specific libraries, effectively utilizing GPUs with massively parallel engines remains difficult. However, using LLMs in practice faces two major challenges: cloud-based APIs pose risks of code leakage, and local deployment is often computationally expensive and inefficient. These drawbacks have spurred interest in small language models (SLMs), which are more lightweight and privacy-friendly. Encouragingly, recent studies show that SLMs can achieve performance comparable to LLMs on specific tasks. While SLMs can match LLMs on domain-specific tasks, their limited reasoning abilities lead to suboptimal performance in complex CUDA generation according to our experiments. To bridge this gap, we propose ReGraphT, a training-free, retrieval-augmented generation framework that transfers LLM-level reasoning to smaller models. ReGraphT organizes CUDA optimization trajectories into a structured reasoning graph, modeling the combined CUDA optimizations as state transitions, and leverages Monte Carlo Graph Search (MCGS) for efficient exploration. We also present a CUDA-specific benchmark with difficulty tiers defined by reasoning complexity to evaluate models more comprehensively. Experiments show that ReGraphT outperforms HPC-specific fine-tuned models and other retrieval-augmented approaches, achieving an average 2.33 speedup on CUDAEval and ParEval. ReGraphT enables SLMs to approach LLM-level performance without the associated privacy risks or excessive computing overhead. The continuous performance improvement of NVIDIA GPUs (Dally et al., 2021; Lindholm et al., 2008; Nickolls & Dally, 2010; Owens et al., 2008) has solidified CUDA as a dominant programming model for high-performance computing tasks, including AI and scientific computing. However, writing efficient CUDA code that fully exploits the massively parallel processing capabilities of GPUs remains a significant challenge.


From Denoising to Refining: A Corrective Framework for Vision-Language Diffusion Model

arXiv.org Artificial Intelligence

Discrete diffusion models have emerged as a promising direction for vision-language tasks, offering bidirectional context modeling and theoretical paralleliza-tion. However, their practical application is severely hindered by a train-inference discrepancy, which leads to catastrophic error cascades: initial token errors during parallel decoding pollute the generation context, triggering a chain reaction of compounding errors and leading to syntactic errors and semantic hallucinations. To address this fundamental challenge, we reframe the generation process from passive denoising to active refining. We introduce ReDiff, a refining-enhanced diffusion framework that teaches the model to identify and correct its own errors. Our approach features a two-stage training process: first, we instill a foundational revision capability by training the model to revise synthetic errors; second, we implement a novel online self-correction loop where the model is explicitly trained to revise its own flawed drafts by learning from an expert's corrections. This mistake-driven learning endows the model with the crucial ability to revisit and refine its already generated output, effectively breaking the error cascade. Extensive experiments demonstrate that ReDiff significantly improves the coherence and factual accuracy of generated content, enabling stable and efficient parallel generation far superior to traditional denoising methods. Our codes and models are available at https://rediff-hku.github.io/. Discrete diffusion models have recently emerged as a promising alternative to the dominant autore-gressive (AR) paradigm for vision-language models (VLMs) (Y ou et al., 2025; Y ang et al., 2025; Li et al., 2025a; Wang et al., 2025a; Swerdlow et al., 2025; Li et al., 2025b; Y u et al., 2025).


Artificial Intelligence Powered Identification of Potential Antidiabetic Compounds in Ficus religiosa

arXiv.org Artificial Intelligence

Diabetes mellitus is a chronic metabolic disorder that necessitates novel therapeutic innovations due to its gradual progression and the onset of various metabolic complications. Research indicates that Ficus religiosa is a conventional medicinal plant that generates bioactive phytochemicals with potential antidiabetic properties. The investigation employs ecosystem - based computational approaches utilizing artificial intelligence to investigate and evaluate compounds de rived from F icus religiosa that exhibit antidiabetic properties. A comprehensive computational procedure incorporated machine learning methodologies, molecular docking techniques, and ADMET prediction systems to assess phytochemical efficacy against the significant antidiabetic enz yme dipeptidyl peptidase - 4 (DPP - 4). Flavonoids and alkaloids have emerged as attractive phytochemicals due to their strong binding int eractions and advantageous pharmacological effects, as indicated by the study. The introduction of AI accelerated screening procedures and enhanced accuracy rates, demonstrating its efficacy in researching plant - based antidiabetic agents. The scientific fo undation now facilitates future experimental validation of natural product therapies tailored for diabetic management. Introduction Type 2 diabetes mellitus is a global metabolic illness characterized by persistent hyperglycemia due to impaired insulin secretion and resistance [1], [2] . T2DM constitutes a growing global health issue, significantly burdening healthcare systems economically [5] . Research in 2021 indicated that 537 million persons globally had diabetes, with projections estimating 643 million cases by 2030 [6] .