Goto

Collaborating Authors

 Law


Low-Resource Neural Machine Translation Using Recurrent Neural Networks and Transfer Learning: A Case Study on English-to-Igbo

arXiv.org Artificial Intelligence

In this study, we develop Neural Machine Translation (NMT) and Transformer-based transfer learning models for English-to-Igbo translation - a low-resource African language spoken by over 40 million people across Nigeria and West Africa. Our models are trained on a curated and benchmarked dataset compiled from Bible corpora, local news, Wikipedia articles, and Common Crawl, all verified by native language experts. We leverage Recurrent Neural Network (RNN) architectures, including Long Short-Term Memory (LSTM) and Gated Recurrent Units (GRU), enhanced with attention mechanisms to improve translation accuracy. To further enhance performance, we apply transfer learning using MarianNMT pre-trained models within the SimpleTransformers framework. Our RNN-based system achieves competitive results, closely matching existing English-Igbo benchmarks. With transfer learning, we observe a performance gain of +4.83 BLEU points, reaching an estimated translation accuracy of 70%. These findings highlight the effectiveness of combining RNNs with transfer learning to address the performance gap in low-resource language translation tasks.


A RAG-Based Multi-Agent LLM System for Natural Hazard Resilience and Adaptation

arXiv.org Artificial Intelligence

Large language models (LLMs) are a transformational capability at the frontier of artificial intelligence and machine learning that can support decision-makers in addressing pressing societal challenges such as extreme natural hazard events. As generalized models, LLMs often struggle to provide context-specific information, particularly in areas requiring specialized knowledge. In this work we propose a retrieval-augmented generation (RAG)-based multi-agent LLM system to support analysis and decision-making in the context of natural hazards and extreme weather events. As a proof of concept, we present WildfireGPT, a specialized system focused on wildfire hazards. The architecture employs a user-centered, multi-agent design to deliver tailored risk insights across diverse stakeholder groups. By integrating natural hazard and extreme weather projection data, observational datasets, and scientific literature through an RAG framework, the system ensures both the accuracy and contextual relevance of the information it provides. Evaluation across ten expert-led case studies demonstrates that WildfireGPT significantly outperforms existing LLM-based solutions for decision support.


What Makes for a Good Saliency Map? Comparing Strategies for Evaluating Saliency Maps in Explainable AI (XAI)

arXiv.org Artificial Intelligence

Saliency maps are a popular approach for explaining classifications of (convolutional) neural networks. However, it remains an open question as to how best to evaluate salience maps, with three families of evaluation methods commonly being used: subjective user measures, objective user measures, and mathematical metrics. We examine three of the most popular saliency map approaches (viz., LIME, Grad-CAM, and Guided Backpropagation) in a between subject study (N=166) across these families of evaluation methods. We test 1) for subjective measures, if the maps differ with respect to user trust and satisfaction; 2) for objective measures, if the maps increase users' abilities and thus understanding of a model; 3) for mathematical metrics, which map achieves the best ratings across metrics; and 4) whether the mathematical metrics can be associated with objective user measures. To our knowledge, our study is the first to compare several salience maps across all these evaluation methods$-$with the finding that they do not agree in their assessment (i.e., there was no difference concerning trust and satisfaction, Grad-CAM improved users' abilities best, and Guided Backpropagation had the most favorable mathematical metrics). Additionally, we show that some mathematical metrics were associated with user understanding, although this relationship was often counterintuitive. We discuss these findings in light of general debates concerning the complementary use of user studies and mathematical metrics in the evaluation of explainable AI (XAI) approaches.


Engineering the Law-Machine Learning Translation Problem: Developing Legally Aligned Models

arXiv.org Artificial Intelligence

Organizations developing machine learning-based (ML) technologies face the complex challenge of achieving high predictive performance while respecting the law. This intersection between ML and the law creates new complexities. As ML model behavior is inferred from training data, legal obligations cannot be operationalized in source code directly. Rather, legal obligations require "indirect" operationalization. However, choosing context-appropriate operationalizations presents two compounding challenges: (1) laws often permit multiple valid operationalizations for a given legal obligation-each with varying degrees of legal adequacy; and, (2) each operationalization creates unpredictable trade-offs among the different legal obligations and with predictive performance. Evaluating these trade-offs requires metrics (or heuristics), which are in turn difficult to validate against legal obligations. Current methodologies fail to fully address these interwoven challenges as they either focus on legal compliance for traditional software or on ML model development without adequately considering legal complexities. In response, we introduce a five-stage interdisciplinary framework that integrates legal and ML-technical analysis during ML model development. This framework facilitates designing ML models in a legally aligned way and identifying high-performing models that are legally justifiable. Legal reasoning guides choices for operationalizations and evaluation metrics, while ML experts ensure technical feasibility, performance optimization and an accurate interpretation of metric values. This framework bridges the gap between more conceptual analysis of law and ML models' need for deterministic specifications. We illustrate its application using a case study in the context of anti-money laundering.


Exploring How LLMs Capture and Represent Domain-Specific Knowledge

arXiv.org Artificial Intelligence

We study whether Large Language Models (LLMs) inherently capture domain-specific nuances in natural language. Our experiments probe the domain sensitivity of LLMs by examining their ability to distinguish queries from different domains using hidden states generated during the prefill phase. We reveal latent domain-related trajectories that indicate the model's internal recognition of query domains. We also study the robustness of these domain representations to variations in prompt styles and sources. Our approach leverages these representations for model selection, mapping the LLM that best matches the domain trace of the input query (i.e., the model with the highest performance on similar traces). Our findings show that LLMs can differentiate queries for related domains, and that the fine-tuned model is not always the most accurate. Unlike previous work, our interpretations apply to both closed and open-ended generative tasks. Large Language Models (LLMs) have demonstrated remarkable capabilities across various tasks, yet the internal mechanisms driving these capabilities remain poorly understood. Different domains require distinct knowledge and reasoning patterns, necessitating LLMs to adjust decision-making based on-the-fly for input queries. This is crucial for applications demanding high reliability, such as legal and medical fields, where errors can lead to significant consequences. The research question of how LLMs adapt their decision-making and reasoning patterns across different domains is distinct from a growing body of work on locating factual associations from language models behavior (Meng et al., 2024; Hernandez et al., 2024a;b; Mitchell et al., 2022; Meng et al., 2023; Dai et al., 2022; Belrose et al., 2023). While these studies aim to identify the modules and computations that recall specific facts, primarily monitoring and controlling language generation, they often fall short in addressing the complexities of generative tasks. Understanding how LLMs adapt their reasoning across generative tasks is important for enhancing transparency in their decision-making processes.


Nemotron-CrossThink: Scaling Self-Learning beyond Math Reasoning

arXiv.org Artificial Intelligence

Large Language Models (LLMs) have shown strong reasoning capabilities, particularly when enhanced through Reinforcement Learning (RL). While prior work has successfully applied RL to mathematical reasoning -- where rules and correctness are well-defined -- generalizing these methods to broader reasoning domains remains challenging due to limited data, the lack of verifiable reward structures, and diverse task requirements. In this work, we propose NEMOTRON-CROSSTHINK, a framework that systematically incorporates multi-domain corpora, including both synthetic and real-world question-answer pairs, into RL training to improve generalization across diverse reasoning tasks. NEMOTRON-CROSSTHINK addresses key challenges by (1) incorporating data from varied sources spanning STEM, humanities, social sciences, etc.; (2) applying structured templates (e.g., multiple-choice and open-ended) to control answer-space complexity; (3) filtering for verifiable answers; and (4) optimizing data blending strategies that utilizes data from multiple sources effectively. Our approach enables scalable and verifiable reward modeling beyond mathematics and demonstrates improved accuracies on both math (MATH-500: +30.1%, AMC23:+27.5%) and non-math reasoning benchmarks (MMLU-PRO: +12.8%, GPQA-DIAMOND: +11.3%, AGIEVAL: +15.1%, SUPERGPQA: +3.8%). Moreover, NEMOTRON-CROSSTHINK exhibits significantly improved response efficiency -- using 28% fewer tokens for correct answers -- highlighting more focused and effective reasoning. Through NEMOTRON-CROSSTHINK, we demonstrate that integrating multi-domain, multi-format data in RL leads to more accurate, efficient, and generalizable LLMs.


California Supreme Court demands State Bar answer questions on AI exam controversy

Los Angeles Times

The California Supreme Court urged the State Bar of California Thursday to explain how and why it utilized artificial intelligence to develop multiple-choice questions for its botched February bar exams. California's highest court, which oversees the State Bar, disclosed Tuesday that its justices were not informed before the exam that the State Bar had allowed its independent psychometrician to use AI to develop a small subset of questions. The Court on Thursday upped its public pressure on the State Bar, demanding it explain how it used AI to develop questions -- and what actions it took to ensure the reliability of the questions. The demand comes as the State Bar petitions the court to adjust test scores for hundreds of prospective California lawyers who complained of multiple technical problems and irregularities during the February exams. Using AI-developed questions written by non-legally-trained psychometricians represents'an obvious conflict of interest,' critics say.


Trump Wants to Blame Fed Chair Powell for Economic Downturn

Slate

This week, Emily Bazelon and David Plotz are joined by Henry Blodget to discuss the financial and political fallout from the President's threats to fire Fed Chair Powell and subsequent retreat; a Supreme Court case over free exercise of religion that could have broad implications; and why Trump stands by Hegseth after Signalgate Part 2. Here are some notes and references from this week's show: Colby Smith for The New York Times: Trump Says He Won't Fire Powell. His Fed Battle May Not Be Over Yet. America's economy is collateral damage Nicole Narea for Vox: Trump's tariffs are driving a gold rush Megan K. Stack for the New York Times (Opinion: Guest Essay): My School District Could Have Avoided This Supreme Court Case Neal McCluskey for Reason: The Supreme Court Is About To Hear 2 Education Cases. Ian Millhiser for Vox: The Supreme Court's "Don't Say Gay" argument went disastrously for public schools Aaron Blake for The Washington Post (Analysis): Even on the gravest of issues, GOP can't summon the will to question Trump Michael Crowley for The New York Times: Critics Call Rubio's Overhaul Plan a Blow to U.S. Values Here are this week's chatters: Henry: Christopher Lamb, Alicia Johnson, Jhasua Razo, and Sarah-Grace Mankarious for CNN: Who will be the next pope?


Google to report earnings amid justice department lawsuits and Trump tariffs

The Guardian

Google's parent company Alphabet will report its first quarter earnings on Thursday, which come as the tech giant is embroiled in antitrust lawsuits brought by the US government and a 17% drop in its stock price since the beginning of the year. It is also the company's first earnings report since Donald Trump levied tariffs on trade partners around the world. Despite the upheaval, analysts appear optimistic on Alphabet's outlook projecting first quarter revenue of 89.2bn, up 11% since the same time last year, and earnings of 2.01 per share, up 7%, according to consensus estimates. Analysts do not expect the global tariffs to create much of an impact for Alphabet, since they were mostly instituted after the end of the quarter. Alphabet is one of the world's most valuable companies, worth nearly 2trn.


OpenAI Wants to Go For-Profit. Experts Say Regulators Should Step In

TIME - Tech

In the latest development in an ongoing struggle over OpenAI's future direction--and potentially the future of artificial intelligence itself--dozens of prominent figures are urging the Attorneys General of California and Delaware to block OpenAI's controversial plan to convert from its unique nonprofit-controlled structure to a for-profit company. In a letter made public April 23, signatories including "AI Godfather" Geoffrey Hinton, Harvard legal professor Lawrence Lessig, and several former OpenAI researchers argue the move represents a fundamental betrayal of OpenAI's founding mission. "The proposed restructuring would eliminate essential safeguards, effectively handing control of, and profits from, what could be the most powerful technology ever created to a for-profit entity with legal duties to prioritize shareholder returns," the letter's authors write. It lands as OpenAI faces immense pressure from the other side: failing to implement the restructure by the end of the year could cost the company 20 billion and hamstring future fundraising. OpenAI was founded in 2015 as a non-profit, with its stated mission being to ensure that artificial general intelligence (AGI) "benefits all of humanity" rather than advancing "the private gain of any person."