Deep Learning
From News to Returns: A Granger-Causal Hypergraph Transformer on the Sphere
Harit, Anoushka, Sun, Zhongtian, Yu, Jongmin
We propose the Causal Sphere Hypergraph Transformer (CSHT), a novel architecture for interpretable financial time-series forecasting that unifies \emph{Granger-causal hypergraph structure}, \emph{Riemannian geometry}, and \emph{causally masked Transformer attention}. CSHT models the directional influence of financial news and sentiment on asset returns by extracting multivariate Granger-causal dependencies, which are encoded as directional hyperedges on the surface of a hypersphere. Attention is constrained via angular masks that preserve both temporal directionality and geometric consistency. Evaluated on S\&P 500 data from 2018 to 2023, including the 2020 COVID-19 shock, CSHT consistently outperforms baselines across return prediction, regime classification, and top-asset ranking tasks. By enforcing predictive causal structure and embedding variables in a Riemannian manifold, CSHT delivers both \emph{robust generalisation across market regimes} and \emph{transparent attribution pathways} from macroeconomic events to stock-level responses. These results suggest that CSHT is a principled and practical solution for trustworthy financial forecasting under uncertainty.
Critical appraisal of artificial intelligence for rare-event recognition: principles and pharmacovigilance case studies
Noren, G. Niklas, Meldau, Eva-Lisa, Ellenius, Johan
Many high-stakes AI applications target low-prevalence events, where apparent accuracy can conceal limited real-world value. Relevant AI models range from expert-defined rules and traditional machine learning to generative LLMs constrained for classification. We outline key considerations for critical appraisal of AI in rare-event recognition, including problem framing and test set design, prevalence-aware statistical evaluation, robustness assessment, and integration into human workflows. In addition, we propose an approach to structured case-level examination (SCLE), to complement statistical performance evaluation, and a comprehensive checklist to guide procurement or development of AI models for rare-event recognition. We instantiate the framework in pharmacovigilance, drawing on three studies: rule-based retrieval of pregnancy-related reports; duplicate detection combining machine learning with probabilistic record linkage; and automated redaction of person names using an LLM. We highlight pitfalls specific to the rare-event setting including optimism from unrealistic class balance and lack of difficult positive controls in test sets - and show how cost-sensitive targets align model performance with operational value. While grounded in pharmacovigilance practice, the principles generalize to domains where positives are scarce and error costs may be asymmetric.
Pitch-Conditioned Instrument Sound Synthesis From an Interactive Timbre Latent Space
Limberg, Christian, Schulz, Fares, Zhang, Zhe, Weinzierl, Stefan
This paper presents a novel approach to neural instrument sound synthesis using a two-stage semi-supervised learning framework capable of generating pitch-accurate, high-quality music samples from an expressive timbre latent space. Existing approaches that achieve sufficient quality for music production often rely on high-dimensional latent representations that are difficult to navigate and provide unintuitive user experiences. We address this limitation through a two-stage training paradigm: first, we train a pitch-timbre disentangled 2D representation of audio samples using a Variational Autoencoder; second, we use this representation as conditioning input for a Transformer-based generative model. The learned 2D latent space serves as an intuitive interface for navigating and exploring the sound landscape. We demonstrate that the proposed method effectively learns a disentangled timbre space, enabling expressive and controllable audio generation with reliable pitch conditioning. Experimental results show the model's ability to capture subtle variations in timbre while maintaining a high degree of pitch accuracy. The usability of our method is demonstrated in an interactive web application, highlighting its potential as a step towards future music production environments that are both intuitive and creatively empowering: https://pgesam.faresschulz.com
Evaluation of Clinical Trials Reporting Quality using Large Language Models
Laรฏ-king, Mathieu, Paroubek, Patrick
Reporting quality is an important topic in clinical trial research articles, as it can impact clinical decisions. In this article, we test the ability of large language models to assess the reporting quality of this type of article using the Consolidated Standards of Reporting Trials (CONSORT). We create CONSORT-QA, an evaluation corpus from two studies on abstract reporting quality with CONSORT-abstract standards. We then evaluate the ability of different large generative language models (from the general domain or adapted to the biomedical domain) to correctly assess CONSORT criteria with different known prompting methods, including Chain-of-thought. Our best combination of model and prompting method achieves 85% accuracy. Using Chain-of-thought adds valuable information on the model's reasoning for completing the task.
Towards Fast Option Pricing PDE Solvers Powered by PIELM
Srinivasan, Akshay Govind, Said, Anuj Jagannath, Pentela, Sathwik, Dwivedi, Vikas, Srinivasan, Balaji
Partial differential equation (PDE) solvers underpin modern quantitative finance, governing option pricing and risk evaluation. Physics-Informed Neural Networks (PINNs) have emerged as a promising approach for solving the forward and inverse problems of partial differential equations (PDEs) using deep learning. However they remain computationally expensive due to their iterative gradient descent based optimization and scale poorly with increasing model size. This paper introduces Physics-Informed Extreme Learning Machines (PIELMs) as fast alternative to PINNs for solving both forward and inverse problems in financial PDEs. PIELMs replace iterative optimization with a single least-squares solve, enabling deterministic and efficient training. We benchmark PIELM on the Black-Scholes and Heston-Hull-White models for forward pricing and demonstrate its capability in inverse model calibration to recover volatility and interest rate parameters from noisy data. From experiments we observe that PIELM achieve accuracy comparable to PINNs while being up to $30\times$ faster, highlighting their potential for real-time financial modeling.
Read the Scene, Not the Script: Outcome-Aware Safety for LLMs
Wu, Rui, Quan, Yihao, Shi, Zeru, Wang, Zhenting, Li, Yanshu, Tang, Ruixiang
Safety-aligned Large Language Models (LLMs) still show two dominant failure modes: they are easily jailbroken, or they over-refuse harmless inputs that contain sensitive surface signals. We trace both to a common cause: current models reason weakly about links between actions and outcomes and over-rely on surface-form signals, lexical or stylistic cues that do not encode consequences. We define this failure mode as Consequence-blindness. To study consequence-blindness, we build a benchmark named CB-Bench covering four risk scenarios that vary whether semantic risk aligns with outcome risk, enabling evaluation under both matched and mismatched conditions which are often ignored by existing safety benchmarks. Mainstream models consistently fail to separate these risks and exhibit consequence-blindness, indicating that consequence-blindness is widespread and systematic. To mitigate consequence-blindness, we introduce CS-Chain-4k, a consequence-reasoning dataset for safety alignment. Models fine-tuned on CS-Chain-4k show clear gains against semantic-camouflage jailbreaks and reduce over-refusal on harmless inputs, while maintaining utility and generalization on other benchmarks. These results clarify the limits of current alignment, establish consequence-aware reasoning as a core alignment goal and provide a more practical and reproducible evaluation path.
Crash Severity Prediction Using Deep Learning Approaches: A Hybrid CNN-RNN Framework
Accurate and timely prediction of crash severity is crucial in mitigating the severe consequences of traffic accidents. Accurate and timely prediction of crash severity is crucial in mitigating the severe consequences of traffic accidents. In order to provide appropriate levels of medical assistance and transportation services, an intelligent transportation system relies on effective prediction methods. Deep learning models have gained popularity in this domain due to their capability to capture non-linear relationships among variables. In this research, we have implemented a hybrid CNN-RNN deep learning model for crash severity prediction and compared its performance against widely used statistical and machine learning models such as logistic regression, naรฏve bayes classifier, K-Nearest Neighbors (KNN), decision tree, and individual deep learning models: RNN and CNN. This study employs a methodology that considers the interconnected relationships between various features of traffic accidents. The study was conducted using a dataset of 15,870 accident records gathered over a period of seven years between 2015 and 2021 on Virginia highway I-64. The findings demonstrate that the proposed CNN-RNN hybrid model has outperformed all benchmark models in terms of predicting crash severity. This result illustrates the effectiveness of the hybrid model as it combines the advantages of both RNN and CNN models in order to achieve greater accuracy in the prediction process.
Measuring Language Model Hallucinations Through Distributional Correctness
Common evaluation paradigms for language models focus on scoring single responses through accuracy metrics or proper scoring rules, failing to capture the full richness of a model's belief state. Recent work illustrates that language models hallucinate in-part because they are optimised to be good test-takers under binary scoring schemes that reward any answer over abstention. While this insight naturally leads to penalty-based approaches, they ignore crucial distinctions in how models distribute uncertainty, for example between hedging toward incorrect answers versus hedging toward "I don't know" responses. A novel evaluation metric, the Distributional Correctness Score (DCS), is introduced to solve this problem, i.e., of not considering a model's entire probability distribution over answer choices. DCS naturally distinguishes between harmful overconfidence in wrong answers and uncertainty expressed through abstention, providing scores in an interpretable default range. Through theoretical analysis and illustrative examples, DCS is demonstrated to offer a more nuanced and aligned evaluation paradigm that incentivises models to express genuine uncertainty rather than guessing. Adapting 12 existing evaluation benchmarks to DCS's variants and measuring performance on six language models reveals that for half of the tested benchmarks scores are negative across all tested models, indicating significant tendencies towards hallucination. Evaluation of language models has commonly focused on whether they produce'correct' or desired outputs in response to given inputs or instructions, as measured using accuracy or probability-based scoring rules that account for confidence in model predictions. However, the paradigm of focusing on a single answer fundamentally misses a critical aspect of evaluating performance: how models distribute their beliefs across the space of possible responses, including the possibility of abstaining from answering in conditions of uncertainty. Recent work (Kalai et al., 2025) provides compelling evidence that language model'hallucinations' persist in-part due to the socio-technical problem of flawed evaluation metrics. Under traditional binary scoring - where correct answers receive a positive score (maximally 1 for perfect correctness), any response like "I don't know" (IDK) receives 0, and incorrect answers also receive 0 - the optimal strategy for any rational agent is to always guess rather than abstain, even when confidence in the guess is minimal. This creates a systematic bias in our evaluation paradigms toward overconfident responses and offers a socio-technical explanation for why language models persist in making confident assertions about uncertain information, i.e., 'hallucinate'.
HoRA: Cross-Head Low-Rank Adaptation with Joint Hypernetworks
Diep, Nghiem T., Le, Dung, Truong, Tuan, Dinh, Tan, Nguyen, Huy, Ho, Nhat
Low-Rank Adaptation (LoRA) is a parameter-efficient fine-tuning (PEFT) technique that adapts large pre-trained models by adding low-rank matrices to their weight updates. However, in the context of fine-tuning multi-head self-attention (MHA), LoRA has been employed to adapt each attention head separately, thereby overlooking potential synergies across different heads. To mitigate this issue, we propose a novel Hyper-shared Low-Rank Adaptation (HoRA) method, which utilizes joint hypernetworks to generate low-rank matrices across attention heads. By coupling their adaptation through a shared generator, HoRA encourages cross-head information sharing, and thus directly addresses the aforementioned limitation of LoRA. By comparing LoRA and HoRA through the lens of hierarchical mixture of experts, our theoretical findings reveal that the latter achieves superior sample efficiency to the former. Furthermore, through extensive experiments across diverse language and vision benchmarks, we demonstrate that HoRA outperforms LoRA and other PEFT methods while requiring only a marginal increase in the number of trainable parameters.
Equipping Retrieval-Augmented Large Language Models with Document Structure Awareness
Xu, Lingnan, Feng, Chong, Zhang, Kaiyuan, Zhengyong, Liu, Xu, Wenqiang, Meng, Fanqing
While large language models (LLMs) demonstrate impressive capabilities, their reliance on parametric knowledge often leads to factual inaccuracies. Retrieval-Augmented Generation (RAG) mitigates this by leveraging external documents, yet existing approaches treat retrieved passages as isolated chunks, ignoring valuable structure that is crucial for document organization. Motivated by this gap, we propose Retrieve-DocumentRoute-Read (RDR2), a novel framework that explicitly incorporates structural information throughout the RAG process. RDR2 employs an LLM-based router to dynamically navigate document structure trees, jointly evaluating content relevance and hierarchical relationships to assemble optimal evidence. Our key innovation lies in formulating document routing as a trainable task, with automatic action curation and structure-aware passage selection inspired by human reading strategies. Through comprehensive evaluation on five challenging datasets, RDR2 achieves state-of-the-art performance, demonstrating that explicit structural awareness significantly enhances RAG systems' ability to acquire and utilize knowledge, particularly in complex scenarios requiring multi-document synthesis.