Industry
On the Analogy between Human Brain and LLMs: Spotting Key Neurons in Grammar Perception
Norouzi, Sanaz Saki, Masjedi, Mohammad, Hitzler, Pascal
Artificial Neural Networks, the building blocks of AI, were inspired by the human brain's network of neurons. Over the years, these networks have evolved to replicate the complex capabilities of the brain, allowing them to handle tasks such as image and language processing. In the realm of Large Language Models, there has been a keen interest in making the language learning process more akin to that of humans. While neuroscientific research has shown that different grammatical categories are processed by different neurons in the brain, we show that LLMs operate in a similar way. Utilizing Llama 3, we identify the most important neurons associated with the prediction of words belonging to different part-of-speech tags. Using the achieved knowledge, we train a classifier on a dataset, which shows that the activation patterns of these key neurons can reliably predict part-of-speech tags on fresh data. The results suggest the presence of a subspace in LLMs focused on capturing part-of-speech tag concepts, resembling patterns observed in lesion studies of the brain in neuroscience.
EASE: Practical and Efficient Safety Alignment for Small Language Models
Shi, Haonan, Wang, Guoli, Ouyang, Tu, Wang, An
Small language models (SLMs) are increasingly deployed on edge devices, making their safety alignment crucial yet challenging. Current shallow alignment methods that rely on direct refusal of malicious queries fail to provide robust protection, particularly against adversarial jailbreaks. While deliberative safety reasoning alignment offers deeper alignment for defending against sophisticated attacks, effectively implanting such reasoning capability in SLMs with limited capabilities remains an open challenge. Moreover, safety reasoning incurs significant computational overhead as models apply reasoning to nearly all queries, making it impractical for resource-constrained edge deployment scenarios that demand rapid responses. We propose EASE, a novel framework that enables practical and Efficient safety Alignment for Small languagE models. Our approach first identifies the optimal safety reasoning teacher that can effectively distill safety reasoning capabilities to SLMs. We then align models to selectively activate safety reasoning for dangerous adversarial jailbreak queries while providing direct responses to straightforward malicious queries and general helpful tasks. This selective mechanism enables small models to maintain robust safety guarantees against sophisticated attacks while preserving computational efficiency for benign interactions. Experimental results demonstrate that EASE reduces jailbreak attack success rates by up to 17% compared to shallow alignment methods while reducing inference overhead by up to 90% compared to deliberative safety reasoning alignment, making it practical for SLMs real-world edge deployments.
A Low-Rank Method for Vision Language Model Hallucination Mitigation in Autonomous Driving
Long, Keke, Guo, Jiacheng, Zhang, Tianyun, Yu, Hongkai, Li, Xiaopeng
Vision Language Models (VLMs) are increasingly used in autonomous driving to help understand traffic scenes, but they sometimes produce hallucinations, which are false details not grounded in the visual input. Detecting and mitigating hallucinations is challenging when ground-truth references are unavailable and model internals are inaccessible. This paper proposes a novel self-contained low-rank approach to automatically rank multiple candidate captions generated by multiple VLMs based on their hallucination levels, using only the captions themselves without requiring external references or model access. By constructing a sentence-embedding matrix and decomposing it into a low-rank consensus component and a sparse residual, we use the residual magnitude to rank captions: selecting the one with the smallest residual as the most hallucination-free. Experiments on the NuScenes dataset demonstrate that our approach achieves 87% selection accuracy in identifying hallucination-free captions, representing a 19% improvement over the unfiltered baseline and a 6-10% improvement over multi-agent debate method. The sorting produced by sparse error magnitudes shows strong correlation with human judgments of hallucinations, validating our scoring mechanism. Additionally, our method, which can be easily parallelized, reduces inference time by 51-67% compared to debate approaches, making it practical for real-time autonomous driving applications.
Learning Time-Varying Graph Signals via Koopman
Krishnan, Sivaram, Choi, Jinho, Park, Jihong
Abstract--A wide variety of real-world data, such as sea measurements, e.g., temperatures collected by distributed sensors and multiple unmanned aerial vehicles (UA V) trajectories, can be naturally represented as graphs, often exhibiting non-Euclidean structures. These graph representations may evolve over time, forming time-varying graphs. Effectively modeling and analyzing such dynamic graph data is critical for tasks like predicting graph evolution and reconstructing missing graph data. In this paper, we propose a framework based on the Koopman autoencoder (KAE) to handle time-varying graph data. Specifically, we assume the existence of a hidden non-linear dynamical system, where the state vector corresponds to the graph embedding of the time-varying graph signals. T o capture the evolving graph structures, the graph data is first converted into a vector time series through graph embedding, representing the structural information in a finite-dimensional latent space. In this latent space, the KAE is applied to learn the underlying non-linear dynamics governing the temporal evolution of graph features, enabling both prediction and reconstruction tasks. A. Motivation Graphs are fundamental data structures for modeling the structure and interactions within complex systems [1] across a variety of domains, including, but not limited to, social networks [2], biological systems [3], transportation networks [4], and communication systems [5]. These data structures provide a versatile framework for representing relationships and dependencies, enabling insights into the organization and behavior of complex systems. In many real-world applications, the underlying graph data is not static; instead they evolve over time. Time-varying graphs [6] are a type of graph data characterized by temporal variations in their components or overall configuration. Unlike the commonly studied static graph structures, analyzing time-varying graph data introduces additional challenges. While the reconstruction of graph signals is necessary for recovering missing information, which is common in real-world sensor networks or data transmission scenarios, prediction, on the other hand, enables forecasting the future states of the systems and thus supports planning, decision-making, and control in dynamical environments. S. Krishnan and J. Choi are with the School of Electrical and Mechanical Engineering, The University of Adelaide, Australia (Emails:{jinho.choi,sivaram.krishan}@adelaide.edu.au), and J. Park is with the Information Systems Technology and Design Pillar, Singapore University of Technology and Design, Singapore (Email: jihong park@sutd.edu.sg).
Explainable AI For Early Detection Of Sepsis
Thakur, Atharva, Dhumal, Shruti
Department of Multidisciplinary Engineering (AI & DS) Vishwakarma Institute of Technology, Pune, 411037, Maharashtra, India Abstract - Sepsis is a potentially fatal medical disorder that needs to be identified and treated right away to avoid fatalities. It must be quickly identified and treated in order to stop it from developing into severe sepsis, septic shock, and multi-organ failure. Sepsis remains a significant problem for doctors despite advancements in medical technology and treatment methods. The beginning of the disease has been successfully predicted by machine learning models in recent years, but due to their black-box character, it is challenging to interpret these predictions and comprehend the underlying illness mechanisms. In this research, we propose a comprehensible AI method for sepsis analysis that combines machine learning with clinical knowledge and expertise in the domain. Our method allows clinicians to understand and verify the model's predictions based on clinical expertise and preexisting beliefs, in addition to providing precise predictions of the onset of sepsis. Keywords - Sepsis, Artificial Intelligence, Machine Learning, Explainable AI, Sensitivity Analysis I. INTRODUCTION As the world continues to advance in technology, the potential of artificial intelligence (AI) in healthcare is becoming more apparent.
Sim-to-Real Transfer in Deep Reinforcement Learning for Bipedal Locomotion
Bao, Lingfan, Peng, Tianhu, Zhou, Chengxu
Abstract--This chapter addresses the critical challenge of simulation-to-reality (sim-to-real) transfer for deep reinforcement learning (DRL) in bipedal locomotion. The first is to shrink the gap through model-centric strategies that systematically improve the simulator's physical fidelity. The second is to harden the policy, a complementary approach that uses in-simulation robustness training and post-deployment adaptation to make the policy inherently resilient to model inaccuracies. The chapter concludes by synthesizing these philosophies into a strategic framework, providing a clear roadmap for developing and evaluating robust sim-to-real solutions. Bipedal robots, machines that walk on two legs, are compelling platforms for operation in human-centric and natural environments. They can climb stairs, step over irregular obstacles, traverse narrow passages, and access spaces that are impractical for wheeled platforms. Their anthropomorphic form factor also enables natural interaction with tools and infrastructure designed for humans, making them suitable for disaster response, healthcare, logistics, and industrial applications. Bipedal locomotion remains challenging because of its high dimensionality, underactuation, and intermittent contacts. Model-based methods struggle with complex dynamics, whereas deep reinforcement learning (DRL) has achieved impressive simulation results in bipedal locomotion through trial and error. As shown in Figure 1, DRL achieves more robust performance than model-based control, particularly as task complexity increases. Most controllers adopt either end-to-end policies that map observations to actions or hierarchical policies that decouple high-level (HL) intent from low-level (LL) execution. Both approaches perform well in simulation but transfer unreliably to hardware, a limitation known as the sim-to-real gap.
Reconstruction and Secrecy under Approximate Distance Queries
Moran, Shay, Nesterova, Elizaveta
Consider the task of locating an unknown target point using approximate distance queries: in each round, a reconstructor selects a query point and receives a noisy version of its distance to the target. This problem arises naturally in various contexts ranging from localization in GPS and sensor networks to privacy-aware data access, and spans a wide variety of metric spaces. It is relevant from the perspective of both the reconstructor (seeking accurate recovery) and the responder (aiming to limit information disclosure, e.g., for privacy or security reasons). We study this reconstruction game through a learning-theoretic lens, focusing on the rate and limits of the best possible reconstruction error. Our first result provides a tight geometric characterization of the optimal error in terms of the Chebyshev radius, a classical concept from geometry. This characterization applies to all compact metric spaces (in fact, even to all totally bounded spaces) and yields explicit formulas for natural metric spaces. Our second result addresses the asymptotic behavior of reconstruction, distinguishing between pseudo-finite spaces -- where the optimal error is attained after finitely many queries -- and spaces where the approximation curve exhibits nontrivial decay. We characterize pseudo-finiteness for convex Euclidean spaces.
EchoMark: Perceptual Acoustic Environment Transfer with Watermark-Embedded Room Impulse Response
Huang, Chenpei, Yao, Lingfeng, Lee, Kyu In, Zhang, Lan Emily, Chen, Xun, Pan, Miao
Acoustic Environment Matching (AEM) is the task of transferring clean audio into a target acoustic environment, enabling engaging applications such as audio dubbing and auditory immersive virtual reality (VR). Recovering similar room impulse response (RIR) directly from reverberant speech offers more accessible and flexible AEM solution. However, this capability also introduces vulnerabilities of arbitrary ``relocation" if misused by malicious user, such as facilitating advanced voice spoofing attacks or undermining the authenticity of recorded evidence. To address this issue, we propose EchoMark, the first deep learning-based AEM framework that generates perceptually similar RIRs with embedded watermark. Our design tackle the challenges posed by variable RIR characteristics, such as different durations and energy decays, by operating in the latent domain. By jointly optimizing the model with a perceptual loss for RIR reconstruction and a loss for watermark detection, EchoMark achieves both high-quality environment transfer and reliable watermark recovery. Experiments on diverse datasets validate that EchoMark achieves room acoustic parameter matching performance comparable to FiNS, the state-of-the-art RIR estimator. Furthermore, a high Mean Opinion Score (MOS) of 4.22 out of 5, watermark detection accuracy exceeding 99\%, and bit error rates (BER) below 0.3\% collectively demonstrate the effectiveness of EchoMark in preserving perceptual quality while ensuring reliable watermark embedding.
A Risk-Neutral Neural Operator for Arbitrage-Free SPX-VIX Term Structures
We propose ARBITER, a risk-neutral neural operator for learning joint SPX-VIX term structures under no-arbitrage constraints. ARBITER maps market states to an operator that outputs implied volatility and variance curves while enforcing static arbitrage (calendar, vertical, butterfly), Lipschitz bounds, and monotonicity. The model couples operator learning with constrained decoders and is trained with extragradient-style updates plus projection. We introduce evaluation metrics for derivatives term structures (NAS, CNAS, NI, Dual-Gap, Stability Rate) and show gains over Fourier Neural Operator, DeepONet, and state-space sequence models on historical SPX and VIX data. Ablation studies indicate that tying the SPX and VIX legs reduces Dual-Gap and improves NI, Lipschitz projection stabilizes calibration, and selective state updates improve long-horizon generalization. We provide identifiability and approximation results and describe practical recipes for arbitrage-free interpolation and extrapolation across maturities and strikes.
When AI Agents Collude Online: Financial Fraud Risks by Collaborative LLM Agents on Social Platforms
Ren, Qibing, Zheng, Zhijie, Guo, Jiaxuan, Yan, Junchi, Ma, Lizhuang, Shao, Jing
In this work, we study the risks of collective financial fraud in large-scale multi-agent systems powered by large language model (LLM) agents. We investigate whether agents can collaborate in fraudulent behaviors, how such collaboration amplifies risks, and what factors influence fraud success. To support this research, we present MultiAgentFraudBench, a large-scale benchmark for simulating financial fraud scenarios based on realistic online interactions. The benchmark covers 28 typical online fraud scenarios, spanning the full fraud lifecycle across both public and private domains. We further analyze key factors affecting fraud success, including interaction depth, activity level, and fine-grained collaboration failure modes. Finally, we propose a series of mitigation strategies, including adding content-level warnings to fraudulent posts and dialogues, using LLMs as monitors to block potentially malicious agents, and fostering group resilience through information sharing at the societal level. Notably, we observe that malicious agents can adapt to environmental interventions. Our findings highlight the real-world risks of multi-agent financial fraud and suggest practical measures for mitigating them. Code is available at https://github.com/zheng977/MutiAgent4Fraud.