Goto

Collaborating Authors

 Government


A Multi-LLM-Agent-Based Framework for Economic and Public Policy Analysis

arXiv.org Artificial Intelligence

We first evaluate five LLMs' economic decision-making capabilities in solving two-period consumption allocation problems under two distinct scenarios: with explicit utility functions and based on intuitive reasoning. While previous research has often simulated heterogeneity by solely varying prompts, our approach harnesses the inherent variations in analytical capabilities across different LLMs to model agents with diverse cognitive traits. Building on these findings, we construct a Multi-LLM-Agent-Based (MLAB) framework by mapping these LLMs to specific educational groups and corresponding income brackets. Using interest-income taxation as a case study, we demonstrate how the MLAB framework can simulate policy impacts across heterogeneous agents, offering a promising new direction for economic and public policy analysis by leveraging LLMs' human-like reasoning capabilities and computational power.


Superintelligent Agents Pose Catastrophic Risks: Can Scientist AI Offer a Safer Path?

arXiv.org Artificial Intelligence

The leading AI companies are increasingly focused on building generalist AI agents -- systems that can autonomously plan, act, and pursue goals across almost all tasks that humans can perform. Despite how useful these systems might be, unchecked AI agency poses significant risks to public safety and security, ranging from misuse by malicious actors to a potentially irreversible loss of human control. We discuss how these risks arise from current AI training methods. Indeed, various scenarios and experiments have demonstrated the possibility of AI agents engaging in deception or pursuing goals that were not specified by human operators and that conflict with human interests, such as self-preservation. Following the precautionary principle, we see a strong need for safer, yet still useful, alternatives to the current agency-driven trajectory. Accordingly, we propose as a core building block for further advances the development of a non-agentic AI system that is trustworthy and safe by design, which we call Scientist AI. This system is designed to explain the world from observations, as opposed to taking actions in it to imitate or please humans. It comprises a world model that generates theories to explain data and a question-answering inference machine. Both components operate with an explicit notion of uncertainty to mitigate the risks of overconfident predictions. In light of these considerations, a Scientist AI could be used to assist human researchers in accelerating scientific progress, including in AI safety. In particular, our system can be employed as a guardrail against AI agents that might be created despite the risks involved. Ultimately, focusing on non-agentic AI may enable the benefits of AI innovation while avoiding the risks associated with the current trajectory. We hope these arguments will motivate researchers, developers, and policymakers to favor this safer path.


HiFi-KPI: A Dataset for Hierarchical KPI Extraction from Earnings Filings

arXiv.org Artificial Intelligence

The U.S. Securities and Exchange Commission (SEC) requires that public companies file financial reports tagging numbers with the machine readable inline eXtensible Business Reporting Language (iXBRL) standard. However, the highly complex and highly granular taxonomy defined by iXBRL limits label transferability across domains. In this paper, we introduce the Hierarchical Financial Key Performance Indicator (HiFi-KPI) dataset, designed to facilitate numerical KPI extraction at specified levels of granularity from unstructured financial text. Our approach organizes a 218,126-label hierarchy using a taxonomy based grouping method, investigating which taxonomy layer provides the most meaningful structure. HiFi-KPI comprises ~1.8M paragraphs and ~5M entities, each linked to a label in the iXBRL-specific calculation and presentation taxonomies. We provide baselines using encoder-based approaches and structured extraction using Large Language Models (LLMs). To simplify LLM inference and evaluation, we additionally release HiFi-KPI Lite, a manually curated subset with four expert-mapped labels. We publicly release all artifacts.


A Meta-Evaluation of Style and Attribute Transfer Metrics

arXiv.org Artificial Intelligence

LLMs make it easy to rewrite text in any style, be it more polite, persuasive, or more positive. We present a large-scale study of evaluation metrics for style and attribute transfer with a focus on content preservation; meaning content not attributed to the style shift is preserved. The de facto evaluation approach uses lexical or semantic similarity metrics often between source sentences and rewrites. While these metrics are not designed to distinguish between style or content differences, empirical meta-evaluation shows a reasonable correlation to human judgment. In fact, recent works find that LLMs prompted as evaluators are only comparable to semantic similarity metrics, even though intuitively, the LLM approach should better fit the task. To investigate this discrepancy, we benchmark 8 metrics for evaluating content preservation on existing datasets and additionally construct a new test set that better aligns with the meta-evaluation aim. Indeed, we then find that the empirical conclusion aligns with the intuition: content preservation metrics for style/attribute transfer must be conditional on the style shift. To support this, we propose a new efficient zero-shot evaluation method using the likelihood of the next token. We hope our meta-evaluation can foster more research on evaluating content preservation metrics, and also to ensure fair evaluation of methods for conducting style transfer.


An Overview of Large Language Models for Statisticians

arXiv.org Machine Learning

Large Language Models (LLMs) have emerged as transformative tools in artificial intelligence (AI), exhibiting remarkable capabilities across diverse tasks such as text generation, reasoning, and decision-making. While their success has primarily been driven by advances in computational power and deep learning architectures, emerging problems -- in areas such as uncertainty quantification, decision-making, causal inference, and distribution shift -- require a deeper engagement with the field of statistics. This paper explores potential areas where statisticians can make important contributions to the development of LLMs, particularly those that aim to engender trustworthiness and transparency for human users. Thus, we focus on issues such as uncertainty quantification, interpretability, fairness, privacy, watermarking and model adaptation. We also consider possible roles for LLMs in statistical analysis. By bridging AI and statistics, we aim to foster a deeper collaboration that advances both the theoretical foundations and practical applications of LLMs, ultimately shaping their role in addressing complex societal challenges.


Advancing Eurasia Fire Understanding Through Machine Learning Techniques

arXiv.org Machine Learning

Modern fire management systems increasingly rely on satellite data and weather forecasting; however, access to comprehensive datasets remains limited due to proprietary restrictions. Despite the ecological significance of wildfires, large-scale, multi-regional research is constrained by data scarcity. Russian diverse ecosystems play a crucial role in shaping Eurasian fire dynamics, yet they remain underexplored. This study addresses existing gaps by introducing an open-access dataset that captures detailed fire incidents alongside corresponding meteorological conditions. We present one of the most extensive datasets available for wildfire analysis in Russia, covering 13 consecutive months of observations. Leveraging machine learning techniques, we conduct exploratory data analysis and develop predictive models to identify key fire behavior patterns across different fire categories and ecosystems. Our results highlight the critical influence of environmental factor patterns on fire occurrence and spread behavior. By improving the understanding of wildfire dynamics in Eurasia, this work contributes to more effective, data-driven approaches for proactive fire management in the face of evolving environmental conditions.


Convergence of Shallow ReLU Networks on Weakly Interacting Data

arXiv.org Machine Learning

Understanding the properties of models used in machine learning is crucial for providing guarantees to downstream users. Of particular importance, the convergence of the training process under gradient methods stands as one of the first issues to address in order to comprehend them. If, on the one hand, such a question for linear models and convex optimization problems (Bottou et al., 2018; Bach, 2024) are well understood, this is not the case for neural networks, which are the most used models in large-scale machine learning. This paper focuses on providing quantitative convergence guarantees for a one-hidden-layer neural network. Theoretically, such global convergence analysis of neural networks has seen two main achievements in the past years: (i) the identification of the lazy regime, due to a particular initialization, where convergence is always guaranteed at the cost of being essentially a linear model (Jacot et al., 2018; Arora et al., 2019; Chizat et al., 2019), and (ii) the proof that with an infinite amount of hidden units a two-layer neural network converges towards the global minimizer of the loss (Mei et al., 2018; Chizat and Bach, 2018; Rotskoff and Vanden-Eijnden, 2018). However, neural networks are trained in practice outside of these regimes, as neural networks are known to perform feature learning, and experimentally reach global minimum with a large but finite number of neurons. Quantifying in which regimes neural networks converge to a global minimum of their loss is still an important open question. 1


Ukraine's Zelenskyy offers to leave office in exchange for peace, NATO

Al Jazeera

Ukrainian President Volodymyr Zelenskyy has offered to give up his position in exchange for peace and membership in NATO as his country was hit by what he called the largest Russian drone attack it has faced since the war began. Zelenskyy made the offer in a news conference in Kyiv on Sunday as the rift widens between himself and the new government in the United States, led by President Donald Trump, on the eve of the three-year anniversary of the beginning of Russia's full-scale war on Ukraine. "If [it means] peace for Ukraine, if you really need me to leave my post, I am ready," Zelenskyy said. "I can exchange this for NATO [membership], if that condition is there, immediately." In a sharp change from the previous US administration of President Joe Biden, which strongly backed Zelenskyy and Ukraine, Trump has increasingly attacked the Ukrainian president since taking office last month.


Trump right to engage Putin on peace talks, says minister

BBC News

US President Donald Trump was right to re-establish links with Russian leader Vladimir Putin to set up peace talks to end the war in Ukraine, a senior Labour minister has said. Education Secretary Bridget Phillipson said there could be "no negotiated peace without Russia" and that Trump's approach had brought "Russians to the table". The US president has faced a backlash for excluding Ukraine from talks after his aides met Russian officials in Saudi Arabia this week. Trump has also suggested Ukraine may be a bystander, saying it has "no cards" in the deal. Prime Minister Sir Keir Starmer will meet Trump in Washington this week and press for Ukraine to be "at the heart" of any peace talks.


Newsom launches new digital democracy tool with initial focus on wildfire victims

Los Angeles Times

California Gov. Gavin Newsom on Sunday announced a new digital democracy initiative that will attempt to connect residents directly with government officials in times of disaster and allow them to express their concerns about matters affecting their day-to-day lives. The web-based initiative, called Engaged California, will go live with a focus on aiding victims of the deadly wildfires in Pacific Palisades and Altadena who are struggling to recover. For example, comments shared via the online forum could potentially prompt government action regarding insurance coverage, building standards or efforts to require utilities to bury power lines underground. In a written statement, Newsom described the pilot program as "a town hall for the modern era -- where Californians share their perspectives, concerns, and ideas geared toward finding real solutions." "We're starting this effort by more directly involving Californians in the LA firestorm response and recovery," he added.