Government
Engineering the Law-Machine Learning Translation Problem: Developing Legally Aligned Models
Hanson, Mathias, Lewkowicz, Gregory, Verboven, Sam
Organizations developing machine learning-based (ML) technologies face the complex challenge of achieving high predictive performance while respecting the law. This intersection between ML and the law creates new complexities. As ML model behavior is inferred from training data, legal obligations cannot be operationalized in source code directly. Rather, legal obligations require "indirect" operationalization. However, choosing context-appropriate operationalizations presents two compounding challenges: (1) laws often permit multiple valid operationalizations for a given legal obligation-each with varying degrees of legal adequacy; and, (2) each operationalization creates unpredictable trade-offs among the different legal obligations and with predictive performance. Evaluating these trade-offs requires metrics (or heuristics), which are in turn difficult to validate against legal obligations. Current methodologies fail to fully address these interwoven challenges as they either focus on legal compliance for traditional software or on ML model development without adequately considering legal complexities. In response, we introduce a five-stage interdisciplinary framework that integrates legal and ML-technical analysis during ML model development. This framework facilitates designing ML models in a legally aligned way and identifying high-performing models that are legally justifiable. Legal reasoning guides choices for operationalizations and evaluation metrics, while ML experts ensure technical feasibility, performance optimization and an accurate interpretation of metric values. This framework bridges the gap between more conceptual analysis of law and ML models' need for deterministic specifications. We illustrate its application using a case study in the context of anti-money laundering.
SCRAG: Social Computing-Based Retrieval Augmented Generation for Community Response Forecasting in Social Media Environments
Sun, Dachun, Lyu, You, Li, Jinning, Chen, Yizhuo, Wang, Tianshi, Kimura, Tomoyoshi, Abdelzaher, Tarek
This paper introduces SCRAG, a prediction framework inspired by social computing, designed to forecast community responses to real or hypothetical social media posts. SCRAG can be used by public relations specialists (e.g., to craft messaging in ways that avoid unintended misinterpretations) or public figures and influencers (e.g., to anticipate social responses), among other applications related to public sentiment prediction, crisis management, and social what-if analysis. While large language models (LLMs) have achieved remarkable success in generating coherent and contextually rich text, their reliance on static training data and susceptibility to hallucinations limit their effectiveness at response forecasting in dynamic social media environments. SCRAG overcomes these challenges by integrating LLMs with a Retrieval-Augmented Generation (RAG) technique rooted in social computing. Specifically, our framework retrieves (i) historical responses from the target community to capture their ideological, semantic, and emotional makeup, and (ii) external knowledge from sources such as news articles to inject time-sensitive context. This information is then jointly used to forecast the responses of the target community to new posts or narratives. Extensive experiments across six scenarios on the X platform (formerly Twitter), tested with various embedding models and LLMs, demonstrate over 10% improvements on average in key evaluation metrics. A concrete example further shows its effectiveness in capturing diverse ideologies and nuances. Our work provides a social computing tool for applications where accurate and concrete insights into community responses are crucial.
A Framework for the Assurance of AI-Enabled Systems
Kapusta, Ariel S., Jin, David, Teague, Peter M., Houston, Robert A., Elliott, Jonathan B., Park, Grace Y., Holdren, Shelby S.
The United States Department of Defense (DOD) looks to accelerate the development and deployment of AI capabilities across a wide spectrum of defense applications to maintain strategic advantages. However, many common features of AI algorithms that make them powerful, such as capacity for learning, large-scale data ingestion, and problem-solving, raise new technical, security, and ethical challenges. These challenges may hinder adoption due to uncertainty in development, testing, assurance, processes, and requirements. Trustworthiness through assurance is essential to achieve the expected value from AI. This paper proposes a claims-based framework for risk management and assurance of AI systems that addresses the competing needs for faster deployment, successful adoption, and rigorous evaluation. This framework supports programs across all acquisition pathways provide grounds for sufficient confidence that an AI-enabled system (AIES) meets its intended mission goals without introducing unacceptable risks throughout its lifecycle. The paper's contributions are a framework process for AI assurance, a set of relevant definitions to enable constructive conversations on the topic of AI assurance, and a discussion of important considerations in AI assurance. The framework aims to provide the DOD a robust yet efficient mechanism for swiftly fielding effective AI capabilities without overlooking critical risks or undermining stakeholder trust.
Exploring How LLMs Capture and Represent Domain-Specific Knowledge
Garcia, Mirian Hipolito, Couturier, Camille, Diaz, Daniel Madrigal, Mallick, Ankur, Kyrillidis, Anastasios, Sim, Robert, Ruhle, Victor, Rajmohan, Saravan
We study whether Large Language Models (LLMs) inherently capture domain-specific nuances in natural language. Our experiments probe the domain sensitivity of LLMs by examining their ability to distinguish queries from different domains using hidden states generated during the prefill phase. We reveal latent domain-related trajectories that indicate the model's internal recognition of query domains. We also study the robustness of these domain representations to variations in prompt styles and sources. Our approach leverages these representations for model selection, mapping the LLM that best matches the domain trace of the input query (i.e., the model with the highest performance on similar traces). Our findings show that LLMs can differentiate queries for related domains, and that the fine-tuned model is not always the most accurate. Unlike previous work, our interpretations apply to both closed and open-ended generative tasks. Large Language Models (LLMs) have demonstrated remarkable capabilities across various tasks, yet the internal mechanisms driving these capabilities remain poorly understood. Different domains require distinct knowledge and reasoning patterns, necessitating LLMs to adjust decision-making based on-the-fly for input queries. This is crucial for applications demanding high reliability, such as legal and medical fields, where errors can lead to significant consequences. The research question of how LLMs adapt their decision-making and reasoning patterns across different domains is distinct from a growing body of work on locating factual associations from language models behavior (Meng et al., 2024; Hernandez et al., 2024a;b; Mitchell et al., 2022; Meng et al., 2023; Dai et al., 2022; Belrose et al., 2023). While these studies aim to identify the modules and computations that recall specific facts, primarily monitoring and controlling language generation, they often fall short in addressing the complexities of generative tasks. Understanding how LLMs adapt their reasoning across generative tasks is important for enhancing transparency in their decision-making processes.
FPGA-Based Neural Network Accelerators for Space Applications: A Survey
Antunes, Pedro, Podobas, Artur
Space missions are becoming increasingly ambitious, necessitating high-performance onboard spacecraft computing systems. In response, field-programmable gate arrays (FPGAs) have garnered significant interest due to their flexibility, cost-effectiveness, and radiation tolerance potential. Concurrently, neural networks (NNs) are being recognized for their capability to execute space mission tasks such as autonomous operations, sensor data analysis, and data compression. This survey serves as a valuable resource for researchers aiming to implement FPGA-based NN accelerators in space applications. By analyzing existing literature, identifying trends and gaps, and proposing future research directions, this work highlights the potential of these accelerators to enhance onboard computing systems.
Trends in AI Supercomputers
Pilz, Konstantin F., Sanders, James, Rahman, Robi, Heim, Lennart
Frontier AI development relies on powerful AI supercomputers, yet analysis of these systems is limited. We create a dataset of 500 AI supercomputers from 2019 to 2025 and analyze key trends in performance, power needs, hardware cost, ownership, and global distribution. We find that the computational performance of AI supercomputers has doubled every nine months, while hardware acquisition cost and power needs both doubled every year. The leading system in March 2025, xAI's Colossus, used 200,000 AI chips, had a hardware cost of \$7B, and required 300 MW of power, as much as 250,000 households. As AI supercomputers evolved from tools for science to industrial machines, companies rapidly expanded their share of total AI supercomputer performance, while the share of governments and academia diminished. Globally, the United States accounts for about 75% of total performance in our dataset, with China in second place at 15%. If the observed trends continue, the leading AI supercomputer in 2030 will achieve $2\times10^{22}$ 16-bit FLOP/s, use two million AI chips, have a hardware cost of \$200 billion, and require 9 GW of power. Our analysis provides visibility into the AI supercomputer landscape, allowing policymakers to assess key AI trends like resource needs, ownership, and national competitiveness.
Latent Representations for Visual Proprioception in Inexpensive Robots
Sheikholeslami, Sahara, Bรถlรถni, Ladislau
Abstract: Robotic manipulation requires explicit or implicit knowledge of the robot's joint positions. Precise proprioception is standard in high-quality industrial robots but is often unavailable in inexpensive robots operating in unstructured environments. In this paper, we ask: to what extent can a fast, single-pass regression architecture perform visual proprioception from a single external camera image, available even in the simplest manipulation settings? We explore several latent representations, including CNNs, V AEs, ViTs, and bags of uncalibrated fiducial markers, using fine-tuning techniques adapted to the limited data available. We evaluate the achievable accuracy through experiments on an inexpensive 6-DoF robot. 1 Introduction Proprioception is the task of recovering the configuration of the robot from its own sensors, in contrast to perception, which is directed towards the external reality. In some settings, proprioception is an engineering problem solved by the internal sensors of the robot. For instance, high-quality industrial robots are so precisely actuated that we can safely consider their joint configurations known. However, for certain scenarios, such as inexpensive robots operating in unstructured environments, the proprioception information coming from the robot might be noisy, uncertain, or unreliable. These robots might be controlled through policies based on end-to-end reinforcement learning or imitation learning that define actions as functions of an external observation a ฯ (o), which appears to sidestep the proprioception problem. In practice, however, if some internal proprioception is available, this can be combined with the results of the external perception, in the hope that explicit proprioceptive data can support task performance.
In a Boon for Tesla, Feds Weaken Rules for Reporting on Self-Driving
Automakers and tech developers testing and deploying self-driving and advanced driver assistance features will no longer have to report as much detailed, public crash information to the federal government, according to a new framework released today by the US Department of Transportation. The moves are a boon for makers of self-driving cars and the wider vehicle technology industry, which has complained that federal crash reporting requirements are overly burdensome and redundant. But the new rules will limit the information available to those who watchdog and study autonomous vehicles and driver assistance features--tech developments that are deeply entwined with public safety but which companies often shield from public view because they involve proprietary systems that companies spend billions to develop. The government's new orders limit "one of the only sources of publicly available data that we have on incidents involving Level 2 systems," says Sam Abuelsamid, who writes about the self-driving vehicle industry and is the vice president of marketing at Telemetry, a Michigan research firm, referring to driver assistance features such as Tesla's Full Self-Driving (Supervised), General Motors' Super Cruise, and Ford's Blue Cruise. These incidents, he notes, are only becoming "more common."
California Supreme Court demands State Bar answer questions on AI exam controversy
The California Supreme Court urged the State Bar of California Thursday to explain how and why it utilized artificial intelligence to develop multiple-choice questions for its botched February bar exams. California's highest court, which oversees the State Bar, disclosed Tuesday that its justices were not informed before the exam that the State Bar had allowed its independent psychometrician to use AI to develop a small subset of questions. The Court on Thursday upped its public pressure on the State Bar, demanding it explain how it used AI to develop questions -- and what actions it took to ensure the reliability of the questions. The demand comes as the State Bar petitions the court to adjust test scores for hundreds of prospective California lawyers who complained of multiple technical problems and irregularities during the February exams. Using AI-developed questions written by non-legally-trained psychometricians represents'an obvious conflict of interest,' critics say.
Trump Wants to Blame Fed Chair Powell for Economic Downturn
This week, Emily Bazelon and David Plotz are joined by Henry Blodget to discuss the financial and political fallout from the President's threats to fire Fed Chair Powell and subsequent retreat; a Supreme Court case over free exercise of religion that could have broad implications; and why Trump stands by Hegseth after Signalgate Part 2. Here are some notes and references from this week's show: Colby Smith for The New York Times: Trump Says He Won't Fire Powell. His Fed Battle May Not Be Over Yet. America's economy is collateral damage Nicole Narea for Vox: Trump's tariffs are driving a gold rush Megan K. Stack for the New York Times (Opinion: Guest Essay): My School District Could Have Avoided This Supreme Court Case Neal McCluskey for Reason: The Supreme Court Is About To Hear 2 Education Cases. Ian Millhiser for Vox: The Supreme Court's "Don't Say Gay" argument went disastrously for public schools Aaron Blake for The Washington Post (Analysis): Even on the gravest of issues, GOP can't summon the will to question Trump Michael Crowley for The New York Times: Critics Call Rubio's Overhaul Plan a Blow to U.S. Values Here are this week's chatters: Henry: Christopher Lamb, Alicia Johnson, Jhasua Razo, and Sarah-Grace Mankarious for CNN: Who will be the next pope?