Government
Graph of Attacks: Improved Black-Box and Interpretable Jailbreaks for LLMs
Akbar-Tajari, Mohammad, Pilehvar, Mohammad Taher, Mahmoody, Mohammad
The challenge of ensuring Large Language Models (LLMs) align with societal standards is of increasing interest, as these models are still prone to adversarial jailbreaks that bypass their safety mechanisms. Identifying these vulnerabilities is crucial for enhancing the robustness of LLMs against such exploits. We propose Graph of ATtacks (GoAT), a method for generating adversarial prompts to test the robustness of LLM alignment using the Graph of Thoughts framework [Besta et al., 2024]. GoAT excels at generating highly effective jailbreak prompts with fewer queries to the victim model than state-of-the-art attacks, achieving up to five times better jailbreak success rate against robust models like Llama. Notably, GoAT creates high-quality, human-readable prompts without requiring access to the targeted model's parameters, making it a black-box attack. Unlike approaches constrained by tree-based reasoning, GoAT's reasoning is based on a more intricate graph structure. By making simultaneous attack paths aware of each other's progress, this dynamic framework allows a deeper integration and refinement of reasoning paths, significantly enhancing the collaborative exploration of adversarial vulnerabilities in LLMs. At a technical level, GoAT starts with a graph structure and iteratively refines it by combining and improving thoughts, enabling synergy between different thought paths. The code for our implementation can be found at: https://github.com/GoAT-pydev/Graph_of_Attacks.
Advanced Longitudinal Control and Collision Avoidance for High-Risk Edge Cases in Autonomous Driving
Chen, Dianwei, Gong, Yaobang, Yang, Xianfeng
Advanced Longitudinal Control and Collision Avoidance for High-Risk Edge Cases in Autonomous Driving Dianwei Chen a, Yaobang Gong a and Xianfeng Terry Yang a, a Department of Civil and Environmental Engineering, University of Maryland, College Park, Maryland, 20740, United StatesA R T I C L E I N F OKeywords: Advanced Driver-Assistance Systems Collision avoidance Reinforcement learning Edge Cases Trajectory calibration model Automatic Emergency Braking A B S T R A C T Advanced Driver-Assistance Systems (ADAS) and Advanced Driving Systems (ADS) are key to improving road safety, yet most existing implementations focus primarily on the vehicle ahead, neglecting the behavior of following vehicles. This shortfall often leads to chain-reaction collisions in high-speed, densely spaced traffic--particularly when a middle vehicle suddenly brakes and trailing vehicles cannot respond in time. To address this critical gap, we propose a novel longitudinal control and collision avoidance algorithm that integrates adaptive cruising with emergency braking. Leveraging deep reinforcement learning, our method simultaneously accounts for both leading and following vehicles. Through a data preprocessing framework that calibrates real-world sensor data, we enhance the robustness and reliability of the training process, ensuring the learned policy can handle diverse driving conditions. In simulated high-risk scenarios (e.g., emergency braking in dense traffic), the algorithm effectively prevents potential pile-up collisions, even in situations involving heavy-duty vehicles. Furthermore, in typical highway scenarios where three vehicles decelerate, the proposed DRL approach achieves a 99% success rate--far surpassing the standard Federal Highway Administration speed concepts guide, which reaches only 36.77% success under the same conditions.1. Introduction Advanced Driver-Assistance Systems (ADASs) and Automated Driving Systems (ADSs) are pivotal technologies in modern vehicles, sharing the overarching goal of improving road safety and paving the way toward fully autonomous driving. ADASs primarily function as semi-automated features that monitor the vehicle's environment, intervening when drivers do not respond adequately (Galvani, 2019; Kukkala et al., 2018). Collectively, these advancements have significantly enhanced road safety and occupant comfort, with many vehicles now including one or more ADAS features as standard equipment. In parallel, efforts in autonomous driving--encompassing levels of automation from partial (Level 3) to fully autonomous (Level 4 or 5)--have led to the development of ADSs (Leiman, 2021). These systems aim to replace or minimize human input in vehicle operation, leveraging advanced sensing, computing, and control technologies to handle dynamic road conditions (Okuda et al., 2014).
Clinical knowledge in LLMs does not translate to human interactions
Bean, Andrew M., Payne, Rebecca, Parsons, Guy, Kirk, Hannah Rose, Ciro, Juan, Mosquera, Rafael, Monsalve, Sara Hincapiรฉ, Ekanayaka, Aruna S., Tarassenko, Lionel, Rocher, Luc, Mahdi, Adam
Global healthcare providers are exploring use of large language models (LLMs) to provide medical advice to the public. LLMs now achieve nearly perfect scores on medical licensing exams, but this does not necessarily translate to accurate performance in real-world settings. We tested if LLMs can assist members of the public in identifying underlying conditions and choosing a course of action (disposition) in ten medical scenarios in a controlled study with 1,298 participants. Participants were randomly assigned to receive assistance from an LLM (GPT-4o, Llama 3, Command R+) or a source of their choice (control). Tested alone, LLMs complete the scenarios accurately, correctly identifying conditions in 94.9% of cases and disposition in 56.3% on average. However, participants using the same LLMs identified relevant conditions in less than 34.5% of cases and disposition in less than 44.2%, both no better than the control group. We identify user interactions as a challenge to the deployment of LLMs for medical advice. Standard benchmarks for medical knowledge and simulated patient interactions do not predict the failures we find with human participants. Moving forward, we recommend systematic human user testing to evaluate interactive capabilities prior to public deployments in healthcare.
Generative to Agentic AI: Survey, Conceptualization, and Challenges
Agentic Artificial Intelligence (AI) builds upon Generative AI (GenAI). It constitutes the next major step in the evolution of AI with much stronger reasoning and interaction capabilities that enable more autonomous behavior to tackle complex tasks. Since the initial release of ChatGPT (3.5), Generative AI has seen widespread adoption, giving users firsthand experience. However, the distinction between Agentic AI and GenAI remains less well understood. To address this gap, our survey is structured in two parts. In the first part, we compare GenAI and Agentic AI using existing literature, discussing their key characteristics, how Agentic AI remedies limitations of GenAI, and the major steps in GenAI's evolution toward Agentic AI. This section is intended for a broad audience, including academics in both social sciences and engineering, as well as industry professionals. It provides the necessary insights to comprehend novel applications that are possible with Agentic AI but not with GenAI. In the second part, we deep dive into novel aspects of Agentic AI, including recent developments and practical concerns such as defining agents. Finally, we discuss several challenges that could serve as a future research agenda, while cautioning against risks that can emerge when exceeding human intelligence.
Technical Challenges in Maintaining Tax Prep Software with Large Language Models
Gogani-Khiabani, Sina, Dewangan, Varsha, Olson, Nina, Trivedi, Ashutosh, Tizpaz-Niari, Saeid
As the US tax law evolves to adapt to ever-changing politico-economic realities, tax preparation software plays a significant role in helping taxpayers navigate these complexities. The dynamic nature of tax regulations poses a significant challenge to accurately and timely maintaining tax software artifacts. The state-of-the-art in maintaining tax prep software is time-consuming and error-prone as it involves manual code analysis combined with an expert interpretation of tax law amendments. We posit that the rigor and formality of tax amendment language, as expressed in IRS publications, makes it amenable to automatic translation to executable specifications (code). Our research efforts focus on identifying, understanding, and tackling technical challenges in leveraging Large Language Models (LLMs), such as ChatGPT and Llama, to faithfully extract code differentials from IRS publications and automatically integrate them with the prior version of the code to automate tax prep software maintenance.
Proof-of-TBI -- Fine-Tuned Vision Language Model Consortium and OpenAI-o3 Reasoning LLM-Based Medical Diagnosis Support System for Mild Traumatic Brain Injury (TBI) Prediction
Gore, Ross, Bandara, Eranga, Shetty, Sachin, Musto, Alberto E., Rana, Pratip, Valencia-Romero, Ambrosio, Rhea, Christopher, Tayebi, Lobat, Richter, Heather, Yarlagadda, Atmaram, Edmonds, Donna, Wallace, Steven, Broshek, Donna
Mild Traumatic Brain Injury (TBI) detection presents significant challenges due to the subtle and often ambiguous presentation of symptoms in medical imaging, making accurate diagnosis a complex task. To address these challenges, we propose Proof-of-TBI, a medical diagnosis support system that integrates multiple fine-tuned vision-language models with the OpenAI-o3 reasoning large language model (LLM). Our approach fine-tunes multiple vision-language models using a labeled dataset of TBI MRI scans, training them to diagnose TBI symptoms effectively. The predictions from these models are aggregated through a consensus-based decision-making process. The system evaluates the predictions from all fine-tuned vision language models using the OpenAI-o3 reasoning LLM, a model that has demonstrated remarkable reasoning performance, to produce the most accurate final diagnosis. The LLM Agents orchestrates interactions between the vision-language models and the reasoning LLM, managing the final decision-making process with transparency, reliability, and automation. This end-to-end decision-making workflow combines the vision-language model consortium with the OpenAI-o3 reasoning LLM, enabled by custom prompt engineering by the LLM agents. The prototype for the proposed platform was developed in collaboration with the U.S. Army Medical Research team in Newport News, Virginia, incorporating five fine-tuned vision-language models. The results demonstrate the transformative potential of combining fine-tuned vision-language model inputs with the OpenAI-o3 reasoning LLM to create a robust, secure, and highly accurate diagnostic system for mild TBI prediction. To the best of our knowledge, this research represents the first application of fine-tuned vision-language models integrated with a reasoning LLM for TBI prediction tasks.
The Big Send-off: High Performance Collectives on GPU-based Supercomputers
Singh, Siddharth, Singh, Mahua, Bhatele, Abhinav
We evaluate the current state of collective communication on GPU-based supercomputers for large language model (LLM) training at scale. Existing libraries such as RCCL and Cray-MPICH exhibit critical limitations on systems such as Frontier -- Cray-MPICH underutilizes network and compute resources, while RCCL suffers from severe scalability issues. To address these challenges, we introduce PCCL, a communication library with highly optimized implementations of all-gather and reduce-scatter operations tailored for distributed deep learning workloads. PCCL is designed to maximally utilize all available network and compute resources and to scale efficiently to thousands of GPUs. It achieves substantial performance improvements, delivering 6-33x speedups over RCCL and 28-70x over Cray-MPICH for all-gather on 2048 GCDs of Frontier. These gains translate directly to end-to-end performance: in large-scale GPT-3-style training, PCCL provides up to 60% and 40% speedups over RCCL for 7B and 13B parameter models, respectively.
Validation and Calibration of Semi-Analytical Models for the Event Horizon Telescope Observations of Sagittarius A*
SaraerToosi, Ali, Broderick, Avery
Fitting ray-traced physical models to EHT observations requires the generation of synthetic images, a task that is computationally demanding. This study leverages ALINet, a generative machine learning model, to efficiently produce radiatively inefficient accretion flow (RIAF) images as a function of the specified physical parameters. ALINet has previously been shown to be able to interpolate black hole images and their associated physical parameters after training on a computationally tractable set of library images. We utilize this model to estimate the uncertainty introduced by a number of anticipated unmodeled physical effects, including interstellar scattering and intrinsic source variability. We then use this to calibrate physical parameter estimates and their associated uncertainties from RIAF model fits to mock EHT data via a library of general relativistic magnetohydrodynamics models.
Explainable Deep-Learning Based Potentially Hazardous Asteroids Classification Using Graph Neural Networks
--Classifying potentially hazardous asteroids (PHAs) is crucial for planetary defense and deep space navigation, yet traditional methods often overlook the dynamical relationships among asteroids. We introduce a Graph Neural Network (GNN) approach that models asteroids as nodes with orbital and physical features, connected by edges representing their similarities, using a NASA dataset of 958,524 records. Despite an extreme class imbalance with only 0.22% of the dataset with hazardous label, our model achieves an overall accuracy of 99% and an AUC of 0.99, with a recall of 78% and an F1-score of 37% for hazardous asteroids after applying Synthetic Minority Oversampling T echnique. Feature importance analysis highlights albedo, perihelion distance, and semi-major axis as main predictors. This framework supports planetary defense missions and confirm AI's potential in enabling autonomous navigation for future missions such as NASA's NEO Surveyor and ESA's Ramses, offering an interpretable and scalable solution for asteroid hazard assessment. However, a small subset known as potentially hazardous asteroids (PHAs) follow orbits that bring them perilously close to our planet, raising the specter of catastrophic collisions. Historical events, such as the 1908 Tunguska explosion [1], which devastated over 2,000 square kilometers of Siberian forest, and the 2013 Chelyabinsk meteor [2], which injured over 1,000 people and caused widespread property damage, show the destructive potential of these celestial bodies.
The Philosophic Turn for AI Agents: Replacing centralized digital rhetoric with decentralized truth-seeking
In the face of rapidly advancing AI technology, individuals will increasingly rely on AI agents to navigate life's growing complexities, raising critical concerns about maintaining both human agency and autonomy. This paper addresses a fundamental dilemma posed by AI decision-support systems: the risk of either becoming overwhelmed by complex decisions, thus losing agency, or having autonomy compromised by externally controlled choice architectures reminiscent of ``nudging'' practices. While the ``nudge'' framework, based on the use of choice-framing to guide individuals toward presumed beneficial outcomes, initially appeared to preserve liberty, at AI-driven scale, it threatens to erode autonomy. To counteract this risk, the paper proposes a philosophic turn in AI design. AI should be constructed to facilitate decentralized truth-seeking and open-ended inquiry, mirroring the Socratic method of philosophical dialogue. By promoting individual and collective adaptive learning, such AI systems would empower users to maintain control over their judgments, augmenting their agency without undermining autonomy. The paper concludes by outlining essential features for autonomy-preserving AI systems, sketching a path toward AI systems that enhance human judgment rather than undermine it.