Large Language Model
An Epidemiological Knowledge Graph extracted from the World Health Organization's Disease Outbreak News
Consoli, Sergio, Coletti, Pietro, Markov, Peter V., Orfei, Lia, Biazzo, Indaco, Schuh, Lea, Stefanovitch, Nicolas, Bertolini, Lorenzo, Ceresa, Mario, Stilianakis, Nikolaos I.
The rapid evolution of artificial intelligence (AI), together with the increased availability of social media and news for epidemiological surveillance, are marking a pivotal moment in epidemiology and public health research. Leveraging the power of generative AI, we use an ensemble approach which incorporates multiple Large Language Models (LLMs) to extract valuable actionable epidemiological information from the World Health Organization (WHO) Disease Outbreak News (DONs). DONs is a collection of regular reports on global outbreaks curated by the WHO and the adopted decision-making processes to respond to them. The extracted information is made available in a daily-updated dataset and a knowledge graph, referred to as eKG, derived to provide a nuanced representation of the public health domain knowledge. We provide an overview of this new dataset and describe the structure of eKG, along with the services and tools used to access and utilize the data that we are building on top. These innovative data resources open altogether new opportunities for epidemiological research, and the analysis and surveillance of disease outbreaks.
Application Of Large Language Models For The Extraction Of Information From Particle Accelerator Technical Documentation
Dai, Qing, Ischebeck, Rasmus, Sapinski, Maruisz, Grycner, Adam
The large set of technical documentation of legacy accelerator systems, coupled with the retirement of experienced personnel, underscores the urgent need for efficient methods to preserve and transfer specialized knowledge. This paper explores the application of large language models (LLMs), to automate and enhance the extraction of information from particle accelerator technical documents. By exploiting LLMs, we aim to address the challenges of knowledge retention, enabling the retrieval of domain expertise embedded in legacy documentation. We present initial results of adapting LLMs to this specialized domain. Our evaluation demonstrates the effectiveness of LLMs in extracting, summarizing, and organizing knowledge, significantly reducing the risk of losing valuable insights as personnel retire. Furthermore, we discuss the limitations of current LLMs, such as interpretability and handling of rare domain-specific terms, and propose strategies for improvement. This work highlights the potential of LLMs to play a pivotal role in preserving institutional knowledge and ensuring continuity in highly specialized fields.
Towards Fundamental Language Models: Does Linguistic Competence Scale with Model Size?
Collado-Montaรฑez, Jaime, Ureรฑa-Lรณpez, L. Alfonso, Montejo-Rรกez, Arturo
Large Language Models offer impressive language capabilities but suffer from well-known limitations, including hallucinations, biases, privacy concerns, and high computational costs. These issues are largely driven by the combination of linguistic competence and factual memorization within a single monolithic model. This paper introduces and empirically supports the Fundamental Language Model (FLM) paradigm, which advocates for smaller, linguistically competent models that offload factual retrieval to external tools. We evaluate models ranging from 135M to 32B parameters across three dimensions: linguistic competence, external factual knowledge, and internal factual knowledge. Our findings reveal that while both linguistic competence and factual knowledge improve with scale, internal factual knowledge grows significantly faster, suggesting that model size is more closely tied to memorization than to core language ability. These results support a modular approach to language modeling, where compact, linguistically proficient models serve as the foundation for tool-augmented systems. The FLM paradigm offers a path toward more efficient, interpretable, and sustainable NLP solutions.
Baichuan-M2: Scaling Medical Capability with Large Verifier System
M2 Team, null, Dou, Chengfeng, Liu, Chong, Yang, Fan, Li, Fei, Jia, Jiyuan, Chen, Mingyang, Ju, Qiang, Wang, Shuai, Dang, Shunya, Li, Tianpeng, Zeng, Xiangrong, Zhou, Yijie, Zhu, Chenzheng, Pan, Da, Deng, Fei, Ai, Guangwei, Dong, Guosheng, Zhang, Hongda, Tai, Jinyang, Hong, Jixiang, Lu, Kai, Sun, Linzhuang, Guo, Peidong, Ma, Qian, Xin, Rihui, Yang, Shihui, Zhang, Shusen, Mo, Yichuan, Liang, Zheng, Zhang, Zhishou, Cui, Hengfu, Zhu, Zuyi, Wang, Xiaochuan
As large language models (LLMs) advance in conversational and reasoning capabilities, their practical application in healthcare has become a critical research focus. However, there is a notable gap between the performance of medical LLMs on static benchmarks such as USMLE and their utility in real-world clinical decision-making. This discrepancy arises because traditional exams fail to capture the dynamic, interactive nature of medical consultations. To address this challenge, we introduce a novel dynamic verification framework that moves beyond static answer verifier, establishing a large-scale, high-fidelity interactive reinforcement learning system. Our framework comprises two key components: a Patient Simulator that creates realistic clinical environments using de-identified medical records, and a Clinical Rubrics Generator that dynamically produces multi-dimensional evaluation metrics. Building on this foundation, we develop Baichuan-M2, a 32B-parameter medical augmented reasoning model trained through a multi-stage reinforcement learning strategy with an improved Group Relative Policy Optimization (GRPO) algorithm. Evaluated on HealthBench, Baichuan-M2 outperforms all other open-source models and most advanced closed-source counterparts, achieving a score above 32 on the challenging HealthBench Hard benchmark-previously exceeded only by GPT-5. Our work demonstrates that robust dynamic verifier system is essential for aligning LLM capabilities with practical clinical applications, establishing a new Pareto front in the performance-parameter trade-off for medical AI deployment.
FActBench: A Benchmark for Fine-grained Automatic Evaluation of LLM-Generated Text in the Medical Domain
Afzal, Anum, Vladika, Juraj, Matthes, Florian
Large Language Models tend to struggle when dealing with specialized domains. While all aspects of evaluation hold importance, factuality is the most critical one. Similarly, reliable fact-checking tools and data sources are essential for hallucination mitigation. We address these issues by providing a comprehensive Fact-checking Benchmark FActBench covering four generation tasks and six state-of-the-art Large Language Models (LLMs) for the Medical domain. We use two state-of-the-art Fact-checking techniques: Chain-of-Thought (CoT) Prompting and Natural Language Inference (NLI). Our experiments show that the fact-checking scores acquired through the Unanimous Voting of both techniques correlate best with Domain Expert Evaluation.
Enhancing Reliability in LLM-Integrated Robotic Systems: A Unified Approach to Security and Safety
Zhang, Wenxiao, Kong, Xiangrui, Dewitt, Conan, Brรคunl, Thomas, Hong, Jin B.
Integrating Large Language Models (LLMs) into robotic systems has revolutionised embodied artificial intelligence, enabling advanced decision-making and adaptability. However, ensuring reliability--encompassing both security against adversarial attacks and safety in complex environments--remains a critical challenge. To address this, we propose a unified framework that mitigates prompt injection attacks while enforcing operational safety through robust validation mechanisms. Our approach combines prompt assembling, state management, and safety validation, evaluated using both performance and security metrics. Experiments show a 30.8% improvement under injection attacks and up to a 325% improvement in complex environment settings under adversarial conditions compared to baseline scenarios. The framework is open-sourced with simulation and physical deployment demos at https://llmeyesim.vercel.app/. Introduction The integration of Large Language Models (LLMs) into embodied robotic systems represents a significant leap in robotic autonomy and adaptability [11]. Recent advances enable robots to interpret natural language instructions, fuse multimodal sensor data, and make planning decisions using the general-purpose reasoning capabilities of models like GPT -4o [22]. These capabilities promise generalist agents that can execute complex, interactive tasks without task-specific training [14]. By drawing on vast internet-scale training corpora, LLMs can produce structured action plans from ambiguous user goals, acting as high-level controllers in dynamic and unpredictable environments [12]. However, these benefits come with risks. Unlike traditional robotic architectures that rely on modular safety subsystems, such as collision avoidance, mission timeouts, and hardware constraints, LLM-based controllers can bypass these safeguards via incorrect inference or adversarial inputs. The semantic sensitivity of LLMs to phrasing, ambiguity, or hallucinated knowledge introduces vulnerabilities not addressed by existing robotics safety protocols [6]. Moreover, integrating multimodal perception (e.g., camera, LiDAR) expands the input space but also introduces new failure modes, where partial, spoofed, or contextually misleading inputs can lead to unsafe behaviours [31]. The current literature lacks a unified methodology to secure and validate the behaviour of LLM-driven robots. Most prior work evaluates vision-language reasoning or robotic planning in isolation and does not consider how prompt injection attacks or input spoofing a ffect downstream physical actions. Similarly, existing LLM safety work focuses on digital assistants or text-only settings, leaving a critical gap in embodied use cases such as autonomous navigation and exploration [37, 20]. As robots begin to operate in open-world human environments, the absence of integrated security and safety layers poses real risks to both mission success and human-robot interaction.
A Theoretical Framework of the Processes of Change in Psychotherapy Delivered by Artificial Agents
Herbener, Arthur Bran, Damholdt, Malene Flensborg
The question of whether artificial agents (e.g., chatbots and social robots) can replace human therapists has received notable attention following the recent launch of large language models. However, little is known about the processes of change in psychotherapy delivered by artificial agents. To facilitate hypothesis development and stimulate scientific debate, the present article offers the first theoretical framework of the processes of change in psychotherapy delivered by artificial agents. The theoretical framework rests upon a conceptual analysis of what active ingredients may be inherently linked to the presence of human therapists. We propose that human therapists' ontological status as human beings and sociocultural status as socially sanctioned healthcare professionals play crucial roles in promoting treatment outcomes. In the absence of the ontological and sociocultural status of human therapists, we propose what we coin the genuineness gap and credibility gap can emerge and undermine key processes of change in psychotherapy. Based on these propositions, we propose avenues for scientific investigations and practical applications aimed at leveraging the strengths of artificial agents and human therapists respectively. We also highlight the intricate agentic nature of artificial agents and discuss how this complicates endeavors to establish universally applicable propositions regarding the processes of change in these interventions.
AMBEDKAR-A Multi-level Bias Elimination through a Decoding Approach with Knowledge Augmentation for Robust Constitutional Alignment of Language Models
Mukhopadhyay, Snehasis, Kasat, Aryan, Dubey, Shivam, Karthikeyan, Rahul, Sood, Dhruv, Jain, Vinija, Chadha, Aman, Das, Amitava
Large Language Models (LLMs) can inadvertently reflect societal biases present in their training data, leading to harmful or prejudiced outputs. In the Indian context, our empirical evaluations across a suite of models reveal that biases around caste and religion are particularly salient. Yet, most existing mitigation strategies are Western-centric and fail to address these local nuances. We propose AMBEDKAR, a framework inspired by the egalitarian vision of Dr B. R. Ambedkar, architect of the Indian Constitution, to guide LLM outputs toward fairness, neutrality, and inclusion in line with Articles 14 to 17. Our approach introduces a Constitution-Aware Decoding Layer, guided by the AI Constitution of India and applied only at inference time, without any parameter updates to the base model. We incorporate a speculative decoding algorithm that proactively reduces casteist and communal bias during generation. This mitigation layer operates directly within the decoding process, avoiding changes to model internals and lowering the computational and infrastructural costs associated with retraining. We reinterpret speculative decoding not merely as an efficiency tool but as a mechanism for fairness. In this framework, a Small Language Model (SLM) acts as a potentially biased generator, while a constitutionally guided Large Language Model (LLM) serves as the verifier. Rather than accelerating generation, the LLM enforces bias-robust trajectories in the SLM outputs. This inversion of roles gives rise to a fairness-by-speculation paradigm. Our approach yields an absolute reduction of bias up to 26.41 percent compared to baseline. Our source code, datasets, and results are available at https://anonymous.4open.science/r/AMBEDKAR-983B/
Scale, Don't Fine-tune: Guiding Multimodal LLMs for Efficient Visual Place Recognition at Test-Time
Cheng, Jintao, Li, Weibin, Luo, Jiehao, Tang, Xiaoyu, He, Zhijian, Wu, Jin, Zou, Yao, Zhang, Wei
Visual Place Recognition (VPR) has evolved from handcrafted descriptors to deep learning approaches, yet significant challenges remain. Current approaches, including Vision Foundation Models (VFMs) and Multimodal Large Language Models (MLLMs), enhance semantic understanding but suffer from high computational overhead and limited cross-domain transferability when fine-tuned. To address these limitations, we propose a novel zero-shot framework employing Test-Time Scaling (TTS) that leverages MLLMs' vision-language alignment capabilities through Guidance-based methods for direct similarity scoring. Our approach eliminates two-stage processing by employing structured prompts that generate length-controllable JSON outputs. The TTS framework with Uncertainty-Aware Self-Consistency (UASC) enables real-time adaptation without additional training costs, achieving superior generalization across diverse environments. Experimental results demonstrate significant improvements in cross-domain VPR performance with up to 210 computational efficiency gains. Keywords: Multimodal Large language Model, Visual Place Recognition, Zero Shot Learning, Autonomous Systems. 1. INTRODUCTION Visual Place Recognition (VPR) represents a cornerstone capability in computer vision and robotics, enabling systems to determine whether a location has been previously visited based solely on visual input.
How Instruction-Tuning Imparts Length Control: A Cross-Lingual Mechanistic Analysis
Rocchetti, Elisabetta, Ferrara, Alfio
Adhering to explicit length constraints, such as generating text with a precise word count, remains a significant challenge for Large Language Models (LLMs). This study aims at investigating the differences between foundation models and their instruction-tuned counterparts, on length-controlled text generation in English and Italian. We analyze both performance and internal component contributions using Cumulative Weighted Attribution, a metric derived from Direct Logit Attribution. Our findings reveal that instruction-tuning substantially improves length control, primarily by specializing components in deeper model layers. Specifically, attention heads in later layers of IT models show increasingly positive contributions, particularly in English. In Italian, while attention contributions are more attenuated, final-layer MLPs exhibit a stronger positive role, suggesting a compensatory mechanism. These results indicate that instruction-tuning reconfigures later layers for task adherence, with component-level strategies potentially adapting to linguistic context.