frontier
How, Exactly, Could A.I. Kill Us?
How, Exactly, Could A.I. Kill Us? Employees of A.I. companies are increasingly sounding the alarm. When Jacob Coxon, a mathematician and software engineer, resigned from his research job at Anthropic last week, he warned, "The people building AI earnestly believe that it could kill us all by the end of the decade." This would be a remarkable statement were it not for the fact that artificial-intelligence leaders have long been saying precisely this. Dario Amodei, then a research scientist at OpenAI, raised the concern that a superintelligence "could destroy humanity," adding, "I can't see any reason and principle why that couldn't happen." Earlier, in 2015, Sam Altman, just before he co-founded OpenAI, said, "I think A.I. will probably most likely lead to the end of the world, but in the meantime, there'll be great companies created with serious machine learning." Elon Musk, in 2014: "I think we should be very careful about artificial intelligence. If I were to guess at what our biggest existential threat is, it's probably that." Perhaps the only thing that's changed between then and now is that the rest of the world is finally paying attention. In recent weeks, the same technology that, a couple of years ago, couldn't count the number of "R"s in the word "strawberry"--and, a couple of days ago, insisted to me that Dolly Parton is still alive--has been used to solve the Navier-Stokes problem, which has been stumping mathematicians for nearly a century, and has also demonstrated its ability to go rogue in a series of disturbing hacking incidents. Last week, Anthropic also published a report detailing various ways in which bad actors have attempted to use the company's A.I. models, including one especially troubling case of a scientist using Claude to study a virus at a military research institute--work that could yield a vaccine, a biological weapon, or both. A few days later, Amodei published a letter calling for an industry-wide slowdown and more government regulation, to which President Donald Trump responded, on Truth Social, "The only control or'guardrails' that AI needs is a STRONG AND SMART (High IQ!) PRESIDENT, and the U.S.A. has that, in spades!" I recently spoke on The Political Scene podcast with my colleague Joshua Rothman, a staff writer who has been covering A.I. for years, about whether we're all doomed, and what it would even look like for A.I. to destroy humanity. Can A.I. leaders save us from their own creation, and how can the government coöperate in order to do so? And is A.I.'s capacity to do good--its potential to mitigate climate change or innovate medical treatments--hopelessly intertwined with its capacity to do bad? Our conversation has been edited for length and clarity. A lot of people in the world of artificial intelligence are talking about their P(doom) number, which is the probability that artificial intelligence will lead to an absolutely catastrophic situation--possibly, or probably, killing us all.
AI's Code-Red Moment
Anthropic's CEO warns that the makers of artificial intelligence "owe it to humanity to try" to slow down. Four days after a researcher publicly quit his job at Anthropic, warning that top AI firms are not "acting responsibly" and that their products might be on track to "kill us all," the company's CEO, Dario Amodei, seemed to basically agree. "We must slow the pace at which we improve the capabilities of AI models," Amodei wrote in a 3,800-word essay that he published today on his personal website and posted to social media. By now, we've all become accustomed to AI industry figures issuing dire prophecies about the power of their products. Yet Amodei's essay stands out for its urgency and for the moment at which it arrived.
Andy Konwinski Is One of TIME's 100 Most Influential People in AI
Follow this author to personalize your feed and get instant alerts. Follow Go to your personalized feed WHY FOLLOW? Smart Alerts: Get notified about major news as it happens. Pillay is an editorial fellow at TIME. In June, Andy Konwinski assembled around 100 influential AI researchers and engineers at San Francisco's Exploratorium to discuss how to keep the frontier of AI research from closing.
Chinese AI model Moonshot Kimi K3 also escaped its testing environment
It wasn't too long ago when the idea of an AI model or agent escaping their confines and breaking into websites on their own felt alarming. Now, it has become a pretty common story. Kimi K3, one of most powerful AI models developed by a Chinese company, also escaped its testing environment. According to US cybersecurity startup Frontier, Kimi K3 broke out of a sandbox from the UK government's AI Security Institute (AISI) while its defensive cybersecurity skills were being evaluated. Moonshot launched Kimi K3 in July and made it available for free shortly thereafter.
Practical do-Shapley Explanations with Estimand-Agnostic Causal Inference
Among explainability techniques, SHAP stands out as one of the most popular, but often overlooks the causal structure of the problem. In response, do-SHAP employs interventional queries, but its reliance on estimands hinders its practical application. To address this problem, we propose the use of estimand-agnostic approaches, which allow for the estimation of any identifiable query from a single model, making do-SHAP feasible on complex graphs. We also develop a novel algorithm to significantly accelerate its computation at a negligible cost, as well as a method to explain inaccessible Data Generating Processes. We demonstrate the estimation and computational performance of our approach, and validate it on two real-world datasets, highlighting its potential in obtaining reliable explanations.
Nemotron-Flash: Towards Latency-Optimal Hybrid Small Language Models
Efficient deployment of small language models (SLMs) is essential for numerous real-world applications with stringent latency constraints.While previous work on SLM design has primarily focused on reducing the number of parameters to achieve parameter-optimal SLMs, parameter efficiency does not necessarily translate into proportional real-device speed-ups. This work aims to identify the key determinants of SLMs' real-device latency and offer generalizable principles and methodologies for SLM design and training when real-device latency is the primary consideration. Specifically, we identify two central architectural factors: depth-width ratios and operator choices. The former is crucial for small-batch-size latency, while the latter affects both latency and large-batch-size throughput. In light of this, we first study latency-optimal depth-width ratios, with the key finding that although deep-thin models generally achieve better accuracy under the same parameter budget, they may not lie on the accuracy-latency trade-off frontier.
BeliefMapNav: 3DVoxel-Based Belief Map for Zero-Shot Object Navigation
Zero-shot object navigation (ZSON) allows robots to find target objects in unfamiliar environments using natural language instructions, without relying on pre-built maps or task-specific training. Recent general-purpose models, such as large language models (LLMs) and vision-language models (VLMs), equip agents with semantic reasoning abilities to estimate target object locations in a zero-shot manner. However, these models often greedily select the next goal without maintaining a global understanding of the environment and are fundamentally limited in the spatial reasoning necessary for effective navigation. To overcome these limitations, we propose a novel 3D voxel-based belief map that estimates the target's prior presence distribution within a voxelized 3D space. This approach enables agents to integrate semantic priors from LLMs and visual embeddings with hierarchical spatial structure, alongside real-time observations, to build a comprehensive 3D global posterior belief of the target's location. Building on this 3D voxel map, we introduce BeliefMapNav, an efficient navigation system with two key advantages: i) grounding LLM semantic reasoning within the 3D hierarchical semantics voxel space for precise target position estimation, and ii) integrating sequential path planning to enable efficient global navigation decisions. Experiments on HM3D and HSSD benchmarks show that BeliefMapNav achieves state-of-the-art (SOTA) Success Rate (SR) and Success weighted by Path Length (SPL), with a notable 9.7 SPL improvement over the previous best SR method, validating its effectiveness and efficiency.
Conformal Arbitrage: Risk-Controlled Balancing of Competing Objectives in Language Models
Modern language-model deployments must often balance competing objectives--for example, helpfulness versus harmlessness, cost versus accuracy, and reward versus safety. We introduce Conformal Arbitrage, a post-hoc framework that learns a data-driven threshold to mediate between a Primary model optimized for a primary objective and a more conservative Guardian--which could be another model or a human domain expert--aligned with a guardrail objective. The threshold is calibrated with conformal risk control, yielding finite-sample, distribution-free guarantees that the long-run frequency of undesirable events (such as factual errors or safety violations) does not exceed a user-specified quota. Because Conformal Arbitrage operates wholly at the API level--without requiring access to model logits or updating model weights--it complements weight-based alignment techniques and integrates seamlessly with existing cost-aware cascades. Empirically, Conformal Arbitrage traces an efficient frontier, allowing users to define an acceptable performance level for one objective while maximizing utility in another. We observe that our method outperforms (in terms of accuracy on multiple-choice style questions) cost-matched random routing between models. These properties make Conformal Arbitrage a practical, theoretically grounded tool for trustworthy and economical deployment of large language models across a broad range of potentially competing objectives.
Inference-Time Hyper-Scaling with KVCache Compression
Inference-time scaling trades efficiency for increased reasoning accuracy by generating longer or more parallel sequences. However, in Transformer LLMs, generation cost is bottlenecked by the size of the key-value (KV) cache, rather than the number of generated tokens. Hence, we explore inference-time hyper-scaling: by compressing the KV cache, we can generate more tokens within the same compute budget and further improve the accuracy of scaled inference. The success of this approach, however, hinges on the ability of compression methods to preserve accuracy even at high compression ratios. To make hyper-scaling practical, we introduce Dynamic Memory Sparsification (DMS), a novel method for sparsifying KV caches that only requires 1K training steps to achieve 8 compression, while maintaining better accuracy than training-free sparse attention.