Goto

Collaborating Authors

 claude opus 4


Nvidia's open Nemotron 3.5 Lightning model is all about specialized, local agentic AI

ZDNet

I wore the world's first HDR10 smart glasses TCL's new E Ink tablet beats the Remarkable and Kindle Anker's new charger is one of the most unique I've ever seen I wore the world's first HDR10 smart glasses TCL's new E Ink tablet beats the Remarkable and Kindle Anker's new charger is one of the most unique I've ever seen Nvidia's open Nemotron 3.5 Lightning model is all about specialized, local agentic AI Our AI Model Release Tracker keeps new models in context with their peers, so you know which are worth your time. AI labs are shipping new models nonstop. Besides being better and faster than their predecessors, not every new model is guaranteed to be a major step change, despite how the company's PR may wax poetic about them. Model strengths really emerge in context: Where are competitor models lacking or excelling? Which models have outstanding specialties, and which are just catching up to industry standards? Our Model Release Tracker helps you make sense of where models stand relative to each other and whether they're worth a deeper look. While we don't test every model or model update on this list, we'll always include the key elements you need to know, along with our hands-on expert test, where applicable.


Anthropic found a hidden space where Claude puzzles over concepts

MIT Technology Review

The AI firm Anthropic has developed a technique that has given it the clearest glimpse yet at what's really going on inside large language models as they answer questions or carry out tasks. What they found ranges from the mundane to the unnerving. Researchers at the company built a tool called the Jacobian lens (or J-lens) and used it to uncover a hidden area, which they named the J-space, inside Claude Opus 4.6, a version of Anthropic's flagship LLM released in February.


Claude Opus 4.8 is learning to say AI's three hardest words: "I don't know"

PCWorld

PCWorld reports that Anthropic's Claude Opus 4.8 focuses on improving AI honesty by teaching the model to admit when it lacks information. The model achieved near-perfect scores in honesty benchmarks for coding questions and exhibited evaluation awareness during testing. Opus 4.8 represents a significant step forward in making AI systems more transparent about their knowledge limitations and uncertainties. Honesty is a key sticking point with even the most powerful LLMs. It's not so much that they're intentionally lying to you; instead, they'll confidently tell you things they're not 100 percent (or even 50 percent) sure about. With Opus 4.8, its latest Claude model, Anthropic says it's made Claude more honest about telling you what it doesn't know, or if it has a low level of confidence in what it's telling you. Released Thursday, Claude Opus 4.8 is Claude Mythos Preview, Anthropic's new "frontier" model that's so powerful, only a handful of "trusted partners" have been allowed to test it for security reasons.


Improving reproducibility by controlling random seed stability in machine learning based estimation via bagging

arXiv.org Machine Learning

Predictions from machine learning algorithms can vary across random seeds, inducing instability in downstream debiased machine learning estimators. We formalize random seed stability via a concentration condition and prove that subbagging guarantees stability for any bounded-outcome regression algorithm. We introduce a new cross-fitting procedure, adaptive cross-bagging, which simultaneously eliminates seed dependence from both nuisance estimation and sample splitting in debiased machine learning. Numerical experiments confirm that the method achieves the targeted level of stability whereas alternatives do not. Our method incurs a small computational penalty relative to standard practice whereas alternative methods incur large penalties.


This is the most misunderstood graph in AI

MIT Technology Review

To some, METR's "time horizon plot" indicates that AI utopia--or apocalypse--is close at hand. The truth is more complicated. Every time OpenAI, Google, or Anthropic drops a new frontier large language model, the AI community holds its breath. It doesn't exhale until METR, an AI research nonprofit whose name stands for "Model Evaluation & Threat Research," updates a now-iconic graph that has played a major role in the AI discourse since it was first released in March of last year. The graph suggests that certain AI capabilities are developing at an exponential rate, and more recent model releases have outperformed that already impressive trend. That was certainly the case for Claude Opus 4.5, the latest version of Anthropic's most powerful model, which was released in late November.


How Claude Code Is Reshaping Software--and Anthropic

WIRED

WIRED spoke with Boris Cherny, head of Claude Code, about how the viral coding tool is changing the way Anthropic works. Engineers in Silicon Valley have been raving about Anthropic's AI coding tool, Claude Code, for months. But recently, the buzz feels as if it's reached a fever pitch. Earlier this week, I sat down with Boris Cherny, head of Claude Code, to try to understand how the company is meeting this moment. "We built the simplest possible thing," said Cherny. "The craziest thing was learning three months ago that half of the sales team at Anthropic uses Claude Code every week."


Why the World's Best AI Systems Are Still So Bad at Pokรฉmon

TIME - Tech

Why the World's Best AI Systems Are Still So Bad at Pokรฉmon Pillay is an editorial fellow at TIME. Pillay is an editorial fellow at TIME. Right now, live on Twitch, you can watch three of the world's smartest AI systems-- GPT 5.2, Claude Opus 4.5, and Gemini 3 Pro --doing their best to beat classic Pokรฉmon games. At least by human standards, they are not very good. The systems are slow, overconfident, and often confused.


Reasoning Models Ace the CFA Exams

arXiv.org Artificial Intelligence

Previous research has reported that large language models (LLMs) demonstrate poor performance on the Chartered Financial Analyst (CFA) exams. However, recent reasoning models have achieved strong results on graduate-level academic and professional examinations across various disciplines. In this paper, we evaluate state-of-the-art reasoning models on a set of mock CFA exams consisting of 980 questions across three Level I exams, two Level II exams, and three Level III exams. Using the same pass/fail criteria from prior studies, we find that most models clear all three levels. The models that pass, ordered by overall performance, are Gemini 3.0 Pro, Gemini 2.5 Pro, GPT-5, Grok 4, Claude Opus 4.1, and DeepSeek-V3.1. Specifically, Gemini 3.0 Pro achieves a record score of 97.6% on Level I. Performance is also strong on Level II, led by GPT-5 at 94.3%. On Level III, Gemini 2.5 Pro attains the highest score with 86.4% on multiple-choice questions while Gemini 3.0 Pro achieves 92.0% on constructed-response questions.


IndiMathBench: Autoformalizing Mathematical Reasoning Problems with a Human Touch

arXiv.org Artificial Intelligence

We introduce IndiMathBench, a human-verified benchmark designed to evaluate mathematical theorem proving, curated using an AI-powered human-assisted pipeline for formalizing natural language problems in Lean. IndiMathBench is composed of 312 formal Lean 4 theorems paired with their corresponding informal problem statements, sourced from Indian Mathematics Olympiads. Through category-based retrieval, iterative compiler feedback, and multi-model ensembles, our pipeline generates candidate formalizations that experts efficiently validate via an interactive dashboard with automated quality summaries. Evaluation across multiple frontier models demonstrates that autoformalization remains challenging, with substantial gaps between syntactic validity and semantic correctness, while theorem proving success rates remain low even with iterative refinement, demonstrating that \benchmark~presents a challenging testbed for mathematical reasoning. IndiMathBench is available at https://github.com/prmbiy/IndiMathBench.


ToolOrchestra: Elevating Intelligence via Efficient Model and Tool Orchestration

arXiv.org Artificial Intelligence

Large language models are powerful generalists, yet solving deep and complex problems such as those of the Humanity's Last Exam (HLE) remains both conceptually challenging and computationally expensive. We show that small orchestrators managing other models and a variety of tools can both push the upper bound of intelligence and improve efficiency in solving difficult agentic tasks. We introduce ToolOrchestra, a method for training small orchestrators that coordinate intelligent tools. ToolOrchestra explicitly uses reinforcement learning with outcome-, efficiency-, and user-preference-aware rewards. Using ToolOrchestra, we produce Orchestrator, an 8B model that achieves higher accuracy at lower cost than previous tool-use agents while aligning with user preferences on which tools are to be used for a given query. On HLE, Orchestrator achieves a score of 37.1%, outperforming GPT-5 (35.1%) while being 2.5x more efficient. On tau2-Bench and FRAMES, Orchestrator surpasses GPT-5 by a wide margin while using only about 30% of the cost. Extensive analysis shows that Orchestrator achieves the best trade-off between performance and cost under multiple metrics, and generalizes robustly to unseen tools. These results demonstrate that composing diverse tools with a lightweight orchestration model is both more efficient and more effective than existing methods, paving the way for practical and scalable tool-augmented reasoning systems.