Claude is helping build its successor, and Anthropic published the gauges

01

WHAT ACTUALLY CHANGED

Anthropic published a prototype R&D Automation Index built from a map of model-development work inside the company. Using Epoch AI's six-level scale, it defines “collaborates” as completing large pieces under close human direction and “leads” as completing most of a task end to end from a high-level prompt while a human supervises. As of August, the company says Claude led 26 percent of measured work, more than 90 percent reached at least collaboration, and no measured subset was fully autonomous.

The reported trajectory is steep. Anthropic's chart places the share led by Claude below 1 percent in February and at 26 percent in August. The underlying basket came from about 15,000 granular tasks extracted from sampled staff work records, organized into a fixed tree with 378 leaf categories, then weighted using person-time as a rough proxy for importance. That is a serious methodology and still a prototype built from company records by company models.

Anthropic also disclosed the scale of its internal agent system. It says approximately 30,000 agents were doing research and engineering work at any one time on the company's most-used internal platform in August. Every action on that platform passed through an online monitor before execution and was also ingested for offline review. The company reports that the online monitor blocked 0.002 percent of more than a billion decisions, or about one in 47,000.

The oversight disclosure includes a human-review funnel. Anthropic says offline monitoring flags roughly 100,000 transcripts each week for additional classifier review, with about 50 highest-priority cases reaching people. Humans review online-blocked actions within one week. These counts describe the monitor's workflow. They do not establish how many harmful actions the monitor missed, how many blocks were false alarms, or how severe the stopped actions would have been.

A third measurement follows compute. For July 13 through July 20, Anthropic says about 6 percent of compute assigned to AI R&D went to safety and about 12 percent of compute assigned to AI-driven AI R&D went to safety. The company calls those estimates conservative and excludes mixed work that advanced safety and capability equally. It also says compute is an imperfect proxy because important safety research can require more human thought than accelerator time.

02

WHY THIS MATTERS

Automation inside an AI laboratory can compress the feedback loop. A model can help write code, run experiments, diagnose defects, analyze results, and prepare the next experiment. If the work is good, a more capable model may arrive sooner and become a stronger research assistant for the following cycle. That does not require a science-fiction system that secretly rewrites itself. A supervised production loop can accelerate long before it becomes autonomous.

The label on the task changes the story. Leading 26 percent of weighted work does not mean Claude made 26 percent of research decisions, produced 26 percent of capability gains, consumed 26 percent of compute, or removed people from 26 percent of jobs. One model-led task can be routine and another decisive. The index needs to remain attached to its task map, weighting rules, evidence, model version, human interventions, and uncertainty.

Monitoring coverage is not detection coverage. Sending every action through a checker is valuable, just as scanning every bag at an airport is valuable. It does not prove the checker recognizes every prohibited item. A rate of one blocked action in 47,000 is therefore neither a safety score nor a misconduct rate. It is the output of one monitor under one policy on one platform, with unknown false negatives and a denominator containing many ordinary actions.

Compute allocation is a governance clue, not a virtue meter. Six percent can be compared over time only if the definitions, workload boundaries, accounting system, and treatment of mixed projects remain stable. It cannot tell readers whether the safety work was good, whether a low-compute insight changed a release, or whether safeguards deployed elsewhere were omitted. The allocation matters most when paired with decisions and outcomes.

Public measurement can become infrastructure. A regular, versioned series from several laboratories, checked by independent evaluators using compatible definitions, would let governments and researchers see whether oversight and safety work keep pace with automated development. It could also expose inconvenient changes that a single launch-day safety report can smooth away. Anthropic has proposed that possibility. Comparable evidence is the test.

03

WHERE IT COULD HELP

• Publish the complete measurement package on a fixed schedule: task taxonomy, weights, sampling method, model and judge versions, human-rating protocol, disagreement intervals, intervention rules, exclusions, revisions, and historical values that never disappear when the dashboard changes
• Give independent evaluators secure access to a representative sample of tasks, source evidence, agent traces, monitor decisions, blocks, human reviews, false positives, known misses, and capability evaluations, with authority to publish disagreements and methods without company approval
• Separate five quantities that are easy to blur: the share of tasks where AI assists, collaborates, or leads; the share of consequential decisions humans retain; the compute used; the measured quality and speed of outputs; and the capability gain attributable to the automated loop
• Stress-test the monitors with planted prohibited actions, novel attack paths, cross-agent coordination, slow-burn misconduct, corrupted context, compromised tools, and deceptive but plausible rationales, then publish detection, false-alarm, latency, severity, and human-response results
• Create a cross-laboratory reporting standard with common definitions, signed data snapshots, third-party calibration, change logs, incident denominators, and release triggers so a rise in automated AI R&D can be compared across time and companies rather than interpreted from marketing language