METR finds Anthropic's models improving about twice as fast, mostly through AI-assisted research

Outside observers are left relying largely on METR's judgment because most of the evidence it found most informative was redacted.

A METR investigation of Anthropic's internal AI research, published Thursday after an eleven-week probe, found the company's models improving roughly twice as fast as a year earlier, with the researchers attributing most of that speed-up to AI assistance rather than simply more computing power. Anthropic staff self-reported being about twelve times as productive with its most capable internal model, and METR saw no clear sign of diminishing returns yet. Three unreleased Anthropic models finished training between July and October, and the lab redacted most of the evidence METR considered most informative, leaving outside observers largely reliant on METR's judgment. The timing matters because Anthropic is preparing to go public while its CEO has urged slowing AI development.