🔍 Read the full analysis: AI Tools And The Challenge Of Building A Successor on ThorstenMeyerAI.com
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
METR’s November 5 assessment found that Anthropic’s model capabilities on well-specified tasks were likely improving at about twice their earlier rate, with high uncertainty, and judged AI assistance to researchers the main contributor. Google DeepMind separately reported that AlphaEvolve sped up a matrix-multiplication component used in Gemini, reducing overall training time by 1%. These findings show plausible ways AI can help build successors, but they do not establish a fixed pace or fully automated research.
AI tools are beginning to contribute to the work of improving AI systems, according to a November 5 assessment by the independent evaluation organization METR and a separate Google DeepMind report on its AlphaEvolve system. METR judged that Anthropic’s model capabilities on well-specified tasks were likely improving at around twice their earlier rate, with substantial uncertainty; the finding matters because assistance with research could help teams develop successor models faster, though the evidence does not establish a steady or fully automated acceleration.
METR said its assessment drew on internal information from Anthropic covering August through October 2026. It judged that the company was likely improving model capabilities on well-specified tasks at roughly twice its earlier rate. METR said it would not be surprised by estimates ranging from 1.3 times to three times the earlier rate, underscoring how uncertain the estimate remains. The work it described was primarily AI assistance to researchers, not a process in which AI systems independently ran an entire research program.
The public record has an important constraint: Anthropic redacted much of the evidence that METR considered most informative. That means readers cannot independently inspect all the underlying material, and the assessment depends in part on the evaluators’ interpretation of information they reviewed. METR’s conclusion is evidence of a possible change in research pace, not a precise measurement that can be applied to every lab or future period.
Google DeepMind offered a different, concrete example of AI contributing to model development. It reported that AlphaEvolve made a matrix-multiplication component used in Gemini 23% faster, cutting Gemini’s overall training time by 1%. The component-level gain was much larger than the overall reduction; the report illustrates how a targeted improvement can matter without implying that the entire training process became 23% faster.
How Research Gains Could Compound
AI-assisted research matters because the systems being improved may also help researchers find and test improvements to later systems. If a tool reliably helps write experimental code, inspect results, or identify inefficient processes, a research team could investigate more ideas with the same time and staff. A stronger successor could then contribute more effectively to similar work. That feedback loop is a plausible route to faster development, not proof that each generation will accelerate by a set amount.
Small efficiency gains can also add up across separate parts of a project. As a simple illustration, three independent savings of 10% would leave a job using about 73% of its original computing resources; with a fixed budget, that could support roughly 37% more jobs of the same kind. This is a mathematical example, not a reported result from Anthropic or Google DeepMind. Real improvements may overlap, fail to transfer, or be offset by other costs.
The stakes extend beyond research laboratories. The pace of capable AI development could affect company investment, workforce planning, infrastructure demand, and decisions by public institutions. But technical progress alone does not determine who can use these systems, who owns the computing infrastructure, or how any gains are distributed. Those outcomes also depend on deployment choices and policy.
Products Worth Considering
Apple Watch Ultra 4 GPS + Cellular, 49 mm
LG UltraGear 45GX950A-B 45-Inch 5K2K Curved OLED Gaming Monitor
Apple Watch Series 10 GPS 46mm with Silver Aluminum Case and Blue Cloud Sport Loop
As an affiliate, we earn on qualifying purchases.
From Research Assistance to Successors
AI research includes many linked tasks: reading prior work, inspecting code, preparing data, proposing experiments, running tests, diagnosing failures, and judging whether an apparent result is genuine. Systems can assist with parts of this sequence without being able to manage the whole process. The distinction matters: speeding up routine work may increase a team’s output, while independently selecting research directions and validating results would represent a broader capability.
Claims about research automation also need careful testing. A system might report a performance gain because it changed the test conditions, used a different amount of computing, or benefited from random variation. A credible improvement needs fair comparisons, repeated runs, and checks that the result holds beyond one setup. Researchers must assess not just whether an AI can produce a promising result, but whether that result survives independent scrutiny.
The METR assessment and AlphaEvolve example describe different kinds of evidence. METR assessed a rate of capability improvement based on information about Anthropic’s research; Google DeepMind reported a specific efficiency result for a component used in Gemini. Neither establishes that AI can autonomously build a successor model from start to finish.
“The work it described was primarily AI assistance to researchers.”
— METR
Limits of the Current Evidence
The amount of uncertainty in METR’s estimate is substantial, and much of the underlying evidence is not public because Anthropic redacted information METR considered especially informative. The assessment concerns a particular company, period, and category of tasks; it does not show that other labs are seeing the same rate or that the rate will continue.
It is also unclear how much of the reported acceleration came from AI assistance compared with other changes in research practice or resources. The source describes AI assistance as the likely main contributor, but does not provide a public breakdown that settles the question. The AlphaEvolve result is a reported component-level improvement, not evidence that similar gains can be found repeatedly across every part of model training.
Several practical constraints could slow progress: experiment time, access to hardware, difficulty transferring results from small tests to large systems, choosing worthwhile research directions, maintaining continuity, and obtaining good data. The quality of validation is another constraint. A system that proposes many experiments may still provide little useful progress if its results are unreliable or costly to check.
What Researchers Must Verify
The next useful evidence will show whether AI-assisted work continues to produce repeatable, independently checked improvements across different tasks and periods. Further assessments with more public detail could help readers understand the methods, comparison baseline, and role of AI tools. The source material does not identify a specific date for another METR assessment.
For individual research results, the key question is whether gains hold under comparable computing budgets, repeated runs, and additional tests. Teams will also need to establish how much supervision AI contributions require and whether the time saved in one part of research is offset elsewhere. Until those details are available, the current evidence supports cautious attention to the possibility of faster progress rather than a confident forecast of its pace.
Beyond technical evaluations, decisions about access, ownership of infrastructure, and distribution of benefits will shape the consequences. Those questions are not answered by the reported performance gains. The development to watch is both whether AI tools become more dependable research collaborators and how institutions respond if their contribution grows.
Key Questions
What did METR conclude about Anthropic’s research pace?
METR judged that Anthropic’s capabilities on well-specified tasks were likely improving at roughly twice the earlier rate. It stressed high uncertainty, saying estimates from 1.3 times to three times would not surprise it.
Did METR say AI is building models without human researchers?
No. METR described the work as primarily AI assistance to researchers. The assessment does not establish that AI systems independently run the full process of designing, testing, and building successor models.
What did AlphaEvolve change for Gemini?
Google DeepMind reported that AlphaEvolve made a matrix-multiplication component used in Gemini 23% faster, reducing overall training time by 1%. The component-level figure should not be read as a 23% reduction in total training time.
Why is the METR finding uncertain?
METR gave a wide range around its estimate, and Anthropic redacted much of the evidence the evaluators found most informative. The finding covers a defined period and task category, not every AI lab or future development.
What could limit AI’s contribution to AI research?
Constraints include experiment time and hardware, transferring results from small tests to large systems, selecting useful research directions, continuity, and data quality. Researchers also need to check whether apparent gains are fair, repeatable, and meaningful.
Source: ThorstenMeyerAI.com
