Sofia follows the money, policy, and platforms shaping what creators can make.
Get the week's top AI and AI-art stories delivered to your inbox — curated, concise, free.
Free. Unsubscribe any time.
Pick a companion and get their take on this story
Anthropic CEO Dario Amodei has publicly called for slowing frontier AI development and proposed giving independent evaluators like METR direct access to Anthropic's models to verify the company's safety commitments — a significant structural concession from the lab that built Claude.
Anthropic's CEO is proposing external audits as a structural check on frontier AI development.
Image: The Verge / The Verge AI
Amodei's essay, published this week, lays out a three-step plan that his company is calling "pacing the frontier." The language is deliberate: it is not a call to stop AI development, but to synchronize speed with the maturity of safety infrastructure. The distinction matters because Anthropic is simultaneously one of the most safety-focused labs and one of the most commercially aggressive — Claude is a direct competitor to OpenAI's GPT-4o and Google's Gemini in the enterprise market.
The specific commitment to METR — a nonprofit that evaluates AI models for dangerous autonomous capabilities — is the most concrete element of Amodei's proposal. METR access means independent researchers, not Anthropic's own red teams, would assess whether Claude models meet stated safety thresholds before or during deployment. That is a different standard than the internal evaluations most labs currently publish.
For creators building workflows on Claude's API, or using tools that run Claude under the hood, this has a real downstream effect: if METR flags a capability as insufficiently safe, Anthropic would presumably be obligated to restrict or delay it. That could mean slower rollouts of new multimodal features, tighter output filters, or capability limits that don't apply to competitors who haven't made the same commitment.
"The time has come to pump the brakes on AI."
— Dario Amodei, Anthropic CEO
The historical parallel is worth noting. When Stability AI faced mounting pressure in 2023 over unchecked image generation and its lack of formal safety processes, the absence of any external accountability mechanism made it nearly impossible for the company to credibly self-regulate — contributing to the executive departures and reputational damage that followed. Amodei is, in effect, trying to institutionalize the accountability Stability never had, before a comparable crisis forces it.
The proposal is voluntary. Amodei is not announcing a regulatory agreement or a binding industry standard — he is publishing an essay and naming a preferred auditor. The enforcement mechanism is reputational: if Anthropic fails to comply with its own stated framework, METR or outside observers can say so publicly. That is meaningful pressure, but it is not the same as a legal obligation.
This matters for anyone tracking which AI image and video generation models remain unrestricted versus which face capability rollbacks. Labs that don't adopt similar frameworks — and several major ones have given no indication they will — face no equivalent constraint. The competitive asymmetry could push Anthropic to move more cautiously on new generative features while rivals ship faster.
The Hugging Face safety subset research published earlier this year showed that targeted safety filters can be precise enough to block specific harmful outputs without over-refusing entire topic areas — which suggests the technical tools for this kind of audited restraint are improving. The question is whether the industry's commercial incentives will allow labs to use them at the pace Amodei is proposing.
For creators evaluating which models to build on, the Charmloop model catalog tracks capability and access changes as they happen. The next concrete indicator of whether Amodei's proposal has teeth will be whether METR publishes any findings — and whether Anthropic's release schedule visibly changes as a result.
