Government
Rethinking the Role of Text Complexity in Language Model Pretraining
Velasco, Dan John, Roque, Matthew Theodore
Improving pretraining data quality and size is known to boost downstream performance, but the role of text complexity--how hard a text is to read--remains less explored. We reduce surface-level complexity (shorter sentences, simpler words, simpler structure) while keeping core content approximately constant and ask: (i) How does complexity affect language modeling across model sizes? (ii) Can useful representations be learned from simpler text alone? (iii) How does pretraining text complexity influence downstream language understanding? We simplify human-written texts using a large language model, pretrain causal models (28M-500M) from scratch on original vs. simplified data, and evaluate them in fine-tuning and zero-shot setups. We find that perplexity is sensitive to the interaction between model capacity and text complexity--smaller models degrade far less on simpler texts--while text complexity has little impact on fine-tuning evaluations, with zero-shot evaluations indicating that simpler texts benefit performance on linguistic knowledge tasks, whereas more complex texts favor tasks requiring world knowledge and entity tracking. Our findings suggest that different types of data diversity affect transfer and zero-shot performance differently, providing insight into tailoring data curation to specific goals.
Reasoning over Boundaries: Enhancing Specification Alignment via Test-time Deliberation
Zhang, Haoran, Li, Yafu, Hu, Xuyang, Liu, Dongrui, Wang, Zhilin, Li, Bo, Cheng, Yu
Large language models (LLMs) are increasingly applied in diverse real-world scenarios, each governed by bespoke behavioral and safety specifications (spec) custom-tailored by users or organizations. These spec, categorized into safety-spec and behavioral-spec, vary across scenarios and evolve with changing preferences and requirements. We formalize this challenge as specification alignment, focusing on LLMs' ability to follow dynamic, scenario-specific spec from both behavioral and safety perspectives. To address this challenge, we propose Align3, a lightweight method that employs Test-Time Deliberation (TTD) with hierarchical reflection and revision to reason over the specification boundaries. We further present SpecBench, a unified benchmark for measuring specification alignment, covering 5 scenarios, 103 spec, and 1,500 prompts. Experiments on 15 reasoning and 18 instruct models with several TTD methods, including Self-Refine, TPO, and MoreThink, yield three key findings: (i) test-time deliberation enhances specification alignment; (ii) Align3 advances the safety-helpfulness trade-off frontier with minimal overhead; (iii) SpecBench effectively reveals alignment gaps. These results highlight the potential of test-time deliberation as an effective strategy for reasoning over the real-world specification boundaries.
GAMA: A General Anonymizing Multi-Agent System for Privacy Preservation Enhanced by Domain Rules and Disproof Mechanism
Yang, Hailong, Zhao, Renhuo, Wang, Guanjin, Deng, Zhaohong
With the rapid advancement of Large Language Models (LLMs), LLM-based agents exhibit exceptional abilities in understanding and generating natural language, enabling human-like collaboration and information transmission in LLM-based Multi-Agent Systems (MAS). High-performance LLMs are often hosted on web servers in public cloud environments. When tasks involve private data, MAS cannot securely utilize these LLMs without implementing the agentic privacy-preserving mechanism. To address this challenge, we propose a General Anonymizing Multi-Agent System (GAMA), which divides the agents' workspace into private and public spaces, ensuring privacy through a structured anonymization mechanism. In the private space, agents handle sensitive data, while in the public web space, only anonymized data is utilized. GAMA incorporates two key modules to mitigate semantic loss caused by anonymization: Domain-Rule-based Knowledge Enhancement (DRKE) and Disproof-based Logic Enhancement (DLE). We evaluate GAMA on two general question-answering datasets, a public privacy leakage benchmark, and two customized question-answering datasets related to privacy. The results demonstrate that GAMA outperforms existing baselines on the evaluated datasets in terms of both task accuracy and privacy preservation metrics.
Local Stability and Region of Attraction Analysis for Neural Network Feedback Systems under Positivity Constraints
Hedesh, Hamidreza Montazeri, Wafi, Moh Kamalul, Siami, Milad
We study the local stability of nonlinear systems in the Lur'e form with static nonlinear feedback realized by feedforward neural networks (FFNNs). By leveraging positivity system constraints, we employ a localized variant of the Aizerman conjecture, which provides sufficient conditions for exponential stability of trajectories confined to a compact set. Using this foundation, we develop two distinct methods for estimating the Region of Attraction (ROA): (i) a less conservative Lyapunov-based approach that constructs invariant sublevel sets of a quadratic function satisfying a linear matrix inequality (LMI), and (ii) a novel technique for computing tight local sector bounds for FFNNs via layer-wise propagation of linear relaxations. These bounds are integrated into the localized Aizerman framework to certify local exponential stability. Numerical results demonstrate substantial improvements over existing integral quadratic constraint-based approaches in both ROA size and scalability.
New Supreme Court term will reshape Trump's powers
New Supreme Court term will reshape Trump's powers The US Supreme Court begins its new term on Monday with a docket already full of potentially significant cases that could define the scope of Donald Trump's presidential authority - and the prospect of more to come. In the eight months that Trump has been back in the White House, he has tested the limits of executive power, unilaterally implementing new policies, slashing federal budgets and workforce, and attempting to bring previously independent agencies and institutions more directly under his control. The latest brewing legal battle comes from the president's attempts to take control of state National Guard units and deploy them in cities where he claims there is public unrest and rampant crime - over the objection of local and state officials. In Oregon, a federal judge has issued orders blocking Trump's deployment of troops to Portland. An appeals court is set to review the move in the coming days.
OpenAI's Blockbuster AMD Deal Is a Bet on Near-Limitless Demand for AI
OpenAI's Blockbuster AMD Deal Is a Bet on Near-Limitless Demand for AI OpenAI's latest move in the race to build massive data centers in the US shows it believes demand for AI will keep surging--even as skeptics warn of a bubble. Sam Altman, CEO of OpenAI, Lisa Su, CEO of Advanced Micro Devices, and Michael Intrator, CEO of CoreWeave, arrive to testify during the Senate on Thursday, May 8, 2025.Photograph: Tom Williams; Getty Images Save this storyOpenAI announced on Monday that it will acquire several data centers' worth of chips from AMD in a blockbuster deal that could also give OpenAI the option to acquire a roughly 10 percent stake in the chipmaker. It's another bold bet from OpenAI that demand for generative artificial intelligence will continue rising--bubble be damned. "Excited to partner with AMD to use their chips to serve our users!" OpenAI CEO Sam Altman said on X, adding that the company will also ramp up its investments in Nvidia chips. He added: "The world needs much more compute " OpenAI said in a blog post this morning that it would commit to purchasing 6 gigawatts' worth of AMD chips over the next several years.
British parts found in Russian drones, Zelensky says
British microcomputers were among more than 100,000 foreign-made parts contained in Russian missiles and drones used in Sunday's deadly strikes on Ukraine, Volodymyr Zelensky has said. The Ukrainian president called for further effective sanctions after saying parts originating in allied countries including Germany, Japan and the US have been identified in Russian weapons. The Department for Business and Trade (DBT) said it had recently undertaken efforts to crack down on UK firms whose products have continued to make their way into Russia's military supply chain. We take reports of goods from UK companies being found in Russian weaponry incredibly seriously, a government spokesperson said. The spokesperson said the government had banned the export of thousands of goods to Russia including every battlefield item Ukraine has brought to our attention, adding that they have imposed the most the most severe package of sanctions. What are the sanctions on Russia and are they working?
Taking These 50 Objects Out of Orbit Would Cut Danger From Space Junk in Half
Old rocket parts and decommissioned satellites are whizzing around in low Earth orbit, where they risk colliding with the ever-growing constellations of modern satellites being launched. A new listing of the 50 most concerning pieces of space debris in low-Earth orbit is dominated by relics more than a quarter-century old, primarily dead rockets left to hurtle through space at the end of their missions. "The things left before 2000 are still the majority of the problem," said Darren McKnight, lead author of a paper presented Friday at the International Astronautical Congress in Sydney. "Seventy-six percent of the objects in the top 50 were deposited last century, and 88 percent of the objects are rocket bodies. That's important to note, especially with some disturbing trends right now."
WIRED Roundup: The New Fake World of OpenAI's Social Video App
On this episode of, we break down some of the week's best stories, covering everything from Peter Thiel's obsession with the Antichrist to the launch of OpenAI's new Sora 2 video app. All products featured on WIRED are independently selected by our editors. However, we may receive compensation from retailers and/or from purchases of products through these links. In today's episode, Zoë Schiffer is joined by WIRED's senior culture editor Manisha Krishnan to run through five of the best stories we published this week--from how federal workers are being told to blame Democrats for the government shutdown to Peter Thiel's ongoing obsession with the Antichrist. Then, Zoë and Manisha break down the news of OpenAI launching a new social app for AI-generated videos. Write to us at uncannyvalley@wired.com . You can always listen to this week's podcast through the audio player on this page, but if you want to subscribe for free to get every episode, here's how: If you're on an iPhone or iPad, open the app called Podcasts, or just tap this link . Today on the show, we're bringing you five stories that you need to know about this week. Including our scoop of how OpenAI just launched a social app dedicated completely to AI-generated videos. I'm joined today by our Senior Culture Editor, Manisha Krishnan. Our first story is about the thing that I feel like our whole newsroom is talking about, possibly the whole country is talking about.