Large Language Model (LLM)-based agents are increasingly deployed in real-world applications such as "digital assistants, autonomous customer service, and decision-support systems", where their ability to "interact in multi-turn, tool-augmented environments" makes them indispensable. However, ensuring the safety of these agents remains a significant challenge due to the diverse and complex risks arising from dynamic user interactions, external tool usage, and the potential for unintended harmful behaviors. To address this critical issue, we propose AutoSafe, the first framework that systematically enhances agent safety through fully automated synthetic data generation. Concretely, 1) we introduce an open and extensible threat model, OTS, which formalizes how unsafe behaviors emerge from the interplay of user instructions, interaction contexts, and agent actions. This enables precise modeling of safety risks across diverse scenarios. 2) we develop a fully automated data generation pipeline that simulates unsafe user behaviors, applies self-reflective reasoning to generate safe responses, and constructs a large-scale, diverse, and high-quality safety training dataset-eliminating the need for hazardous real-world data collection. To evaluate the effectiveness of our framework, we design comprehensive experiments on both synthetic and real-world safety benchmarks. Results demonstrate that AutoSafe boosts safety scores by 45% on average and achieves a 28.91% improvement on real-world tasks, validating the generalization ability of our learned safety strategies. These results highlight the practical advancement and scalability of AutoSafe in building safer LLM-based agents for real-world deployment. We have released the project page at https://auto-safe.github.io/.

We introduce DeepSeek-Prover-V2, an open-source large language model designed for formal theorem proving in Lean 4, with initialization data collected through a recursive theorem proving pipeline powered by DeepSeek-V3. The cold-start training procedure begins by prompting DeepSeek-V3 to decompose complex problems into a series of subgoals. The proofs of resolved subgoals are synthesized into a chain-of-thought process, combined with DeepSeek-V3's step-by-step reasoning, to create an initial cold start for reinforcement learning. This process enables us to integrate both informal and formal mathematical reasoning into a unified model. The resulting model, DeepSeek-Prover-V2-671B, achieves state-of-the-art performance in neural theorem proving, reaching 88.9% pass ratio on the MiniF2F-test and solving 49 out of 658 problems from PutnamBench. In addition to standard benchmarks, we introduce ProverBench, a collection of 325 formalized problems, to enrich our evaluation, including 15 selected problems from the recent AIME competitions (years 24-25). Further evaluation on these 15 AIME problems shows that the model successfully solves 6 of them. In comparison, DeepSeek-V3 solves 8 of these problems using majority voting, highlighting that the gap between formal and informal mathematical reasoning in large language models is substantially narrowing.

Language models (LMs) can memorize and reproduce segments from their pretraining data verbatim even in non-adversarial settings, raising concerns about copyright, plagiarism, privacy, and creativity. We introduce Paraphrase Preference Optimization (ParaPO), a post-training method that fine-tunes LMs to reduce unintentional regurgitation while preserving their overall utility. ParaPO trains LMs to prefer paraphrased versions of memorized segments over the original verbatim content from the pretraining data. To maintain the ability to recall famous quotations when appropriate, we develop a variant of ParaPO that uses system prompts to control regurgitation behavior. In our evaluation on Llama3.1-8B, ParaPO consistently reduces regurgitation across all tested datasets (e.g., reducing the regurgitation metric from 17.3 to 12.9 in creative writing), whereas unlearning methods used in prior work to mitigate regurgitation are less effective outside their targeted unlearned domain (from 17.3 to 16.9). When applied to the instruction-tuned Tulu3-8B model, ParaPO with system prompting successfully preserves famous quotation recall while reducing unintentional regurgitation (from 8.7 to 6.3 in creative writing) when prompted not to regurgitate. In contrast, without ParaPO tuning, prompting the model not to regurgitate produces only a marginal reduction (8.7 to 8.4).

Large language models (LLMs) have recently revolutionized language processing tasks but have also brought ethical and legal issues. LLMs have a tendency to memorize potentially private or copyrighted information present in the training data, which might then be delivered to end users at inference time. When this happens, a naive solution is to retrain the model from scratch after excluding the undesired data. Although this guarantees that the target data have been forgotten, it is also prohibitively expensive for LLMs. Approximate unlearning offers a more efficient alternative, as it consists of ex post modifications of the trained model itself to prevent undesirable results, but it lacks forgetting guarantees because it relies solely on empirical evidence. In this work, we present DP2Unlearning, a novel LLM unlearning framework that offers formal forgetting guarantees at a significantly lower cost than retraining from scratch on the data to be retained. DP2Unlearning involves training LLMs on textual data protected using ε-differential privacy (DP), which later enables efficient unlearning with the guarantees against disclosure associated with the chosen ε. Our experiments demonstrate that DP2Unlearning achieves similar model performance post-unlearning, compared to an LLM retraining from scratch on retained data -- the gold standard exact unlearning -- but at approximately half the unlearning cost. In addition, with a reasonable computational cost, it outperforms approximate unlearning methods at both preserving the utility of the model post-unlearning and effectively forgetting the targeted information.

It's kind of strange that Grok, Elon Musk's chatbot, went full Nazi a few weeks ago, and is just…still out there. But, then again, how are you supposed to hold an LLM accountable? Subscribe to Slate Plus to access ad-free listening to the whole What Next family and all your favorite Slate podcasts. Subscribe today on Apple Podcasts by clicking "Try Free" at the top of our show page. Sign up now at slate.com/whatnextplus to get access wherever you listen.

Mark Zuckerberg was supposed to win the AI race. Eons before ChatGPT and AlphaGo, when OpenAI did not exist and Google had not yet purchased DeepMind, there was FAIR: Facebook AI Research. In 2013, Facebook tapped one of the "godfathers" of AI, the legendary computer scientist Yann LeCun, to lead its new division. That year, Zuckerberg personally traveled to one of the world's most prestigious AI conferences to announce FAIR and recruit top scientists to the lab. FAIR has since made a number of significant contributions to AI research, including in the field of computer vision.

Android Authority has been digging around in the files of the latest ChatGPT app (beta version 1.2025.196) When generating an image with ChatGPT, you will soon be able to select "Save without watermark" in the menu behind the three dots in the top-right corner of the app. Obviously, this feature would be rather useless if images weren't going to be watermarked. Will all users be able to save images without watermarks? Android Authority speculates that the feature may sit behind a paywall and only be available to paid ChatGPT subscribers.

In the early days of large language models, there was a high barrier to entry: it used to be impossible to run anything useful on your own computer without investing in pricey GPUs. But researchers have had so much success in shrinking down and speeding up models that anyone with a laptop, or even a smartphone, can now get in on the action. For people who are concerned about privacy, want to break free from the control of the big LLM companies, or just enjoy tinkering, local models offer a compelling alternative to ChatGPT and its web-based peers. Here's how to get started running a useful model from the safety and comfort of your own computer. This story is part of MIT Technology Review's How To series, helping you get things done.

Mizuho Financial Group said Friday that it has signed a strategic partnership agreement with SoftBank to introduce cutting-edge artificial intelligence to streamline operations and improve customer service. Mizuho will be the first in the financial sector to introduce "Cristal intelligence," which is being developed jointly by SoftBank and OpenAI, the U.S. developer of the ChatGPT generative AI tool. Mizuho expects the latest AI technology, which optimizes corporate tasks, to help the company increase revenue and cut costs, resulting in positive effects totaling 300 billion by fiscal 2030. Using the technology, Mizuho plans to analyze transaction data and market trends to quickly provide corporate customers with management advice. The financial group also expects the technology to help boost productivity in its sales activities more than twofold and reduce low-value operations by up to 50%.

If you've been nodding along in meetings pretending to understand AI tools, now's your chance to actually learn them--no tech background required. This ChatGPT and Automation E-Degree gives you lifetime access to 12 beginner-friendly courses covering over 25 hours of lessons built for real-world use, now on sale for 19.97. You'll start with ChatGPT, learning how to train it for industry-specific tasks, improve communication, and speed up your workday without sacrificing quality. But it doesn't stop at ChatGPT--you'll also explore more than 20 popular AI tools that help you create, automate, and problem-solve like a pro. The curriculum is designed to be hands-on, with practical applications in business, marketing, and even coding.