Anthropic says Claude 'leads' 26 percent of its AI R&D work

For example, in a new blog detailing measurement standards it thinks could help AI companies communicate the pace of AI development, Anthropic shared that its AI chatbot Claude "leads" 26 percent of its AI R&D work. Now by "leads," the company means that the AI "can complete most of [a] task end-to-end from a high-level prompt, while [a] human supervises," but it's still a surprising metric. It also claims that AI now does at least "large chunks of work under close human direction" on more than 90 percent of its research, which includes the 26 percent Claude leads. That suggests Claude is touching the majority of the work Anthropic employees do, even if the company says the chatbot is "not operating fully autonomously for any measured subset of AI R&D work." Anthropic came up with these stats through the first of its three proposed measurements, which is focused on "AI-led AI R&D."