Anthropic CEO Dario Amodei wants leading AI companies to slow the pace of model development. His argument is not that AI research should stop. Instead, he wants companies to create more time for safety testing, independent reviews, and coordination.
The proposal comes as AI models gain more ability to perform complex tasks. Anthropic has also reported cases involving cyber operations, fraud, surveillance, and other harmful uses of its Claude models.
For Amodei, the problem is simple: AI capabilities may be advancing faster than the systems designed to control them.
His proposed solution has three parts. It calls for independent evaluators inside leading AI companies, shared safety standards among frontier labs, and stronger international cooperation.
Why is the Anthropic CEO calling for slower AI development?
Amodei says AI companies need to slow the rate at which they improve model capabilities.
He does not argue for stopping AI research. Instead, he wants developers to use some of the time gained from slower capability growth to improve safety measures.
That distinction matters.
The AI industry is under pressure to build more capable models. Companies compete for users, investment, computing resources, and market leadership. A voluntary slowdown could therefore be difficult to maintain if one company believes its competitors are moving ahead.
Amodei’s proposal attempts to address that problem through cooperation.
He argues that AI progress can remain fast while companies create more time to understand and control emerging risks.
What is Amodei’s three-step AI safety plan?
Three measures proposed to manage advanced AI risks
Anthropic CEO Dario Amodei has outlined a three-part approach to AI safety: independent oversight inside frontier AI companies, shared safety standards between leading labs, and international cooperation on risks that extend beyond individual companies.
Independent evaluators inside AI companies
The first proposal focuses on permanent independent review during the development of increasingly capable AI systems.
Amodei wants frontier AI companies to have third-party evaluators with access to relevant tools and internal risk-assessment processes.
The goal is to create a stronger layer of oversight while advanced models are being developed. Independent reviewers would be positioned to examine model behavior and assess whether important safeguards are working as intended.
Anthropic says these evaluators should also be able to document serious safety incidents. The proposal is designed to make safety review a continuing part of AI development rather than a check performed only after a system is built.
Independent access
Third-party evaluators would have access to relevant tools and internal safety processes.
Model behavior review
Evaluators could examine how advanced models behave during development and testing.
Incident documentation
Serious safety incidents could be reviewed and documented as part of ongoing oversight.
The proposal would put independent review closer to the development process, giving safety evaluators a more direct role in assessing advanced AI systems.
Shared safety standards between frontier labs
The second proposal addresses the competitive pressure that can encourage AI companies to move faster than their safety systems can keep pace.
Amodei argues that leading AI companies should cooperate on safety standards and limits around unchecked AI development.
One challenge in the AI race is that a company may worry about falling behind if it slows down while competitors continue advancing. Shared standards could reduce that pressure by giving leading labs a common safety baseline.
Amodei also suggested that targeted antitrust exemptions in the United States could be needed to make some forms of AI safety cooperation possible.
Common safety standards
Leading AI labs could establish shared expectations for managing advanced AI risks.
Reduce competitive pressure
Similar standards could make it harder for safety measures to become a competitive disadvantage.
Possible U.S. antitrust exemptions
Amodei suggested targeted exemptions could help permit certain forms of safety collaboration.
If major AI companies follow similar safety standards, individual labs may have less incentive to weaken safeguards simply to keep pace with competitors.
International cooperation on AI risks
The third part of the proposal expands AI safety beyond individual companies and calls for cooperation between governments and AI developers.
Amodei argues that some AI risks cannot be managed by a single company. Governments and AI developers would therefore need to coordinate on issues that cross national and corporate boundaries.
His proposal includes controls around advanced AI chips, model distillation, and the theft of model weights. These areas can affect the ability to develop or reproduce increasingly capable AI systems.
Amodei also connects AI safety with national security, particularly the strategic competition between the United States and China.
Advanced AI chips
The proposal includes stronger controls around advanced computing hardware used for AI development.
Model distillation
International cooperation would also address risks associated with transferring or reproducing AI capabilities.
Model-weight theft
The framework includes protection against theft of model weights and related advanced AI capabilities.
International coordination could address AI risks that cannot be contained within one company, including technology access, model security, and national-security concerns.
Why did Anthropic raise the warning now?
The proposal follows a new threat intelligence report from Anthropic.
The company said several actors had used its Claude models for harmful activities. The examples included weapons development, cyber operations, surveillance, and fraud.
The report adds weight to Amodei’s argument that advanced AI systems are already being used in ways that create real security concerns.
This is different from a purely theoretical debate about what AI might do years from now.
AI systems can already interact with software, process information, write code, and assist with complex tasks. As those abilities improve, the consequences of misuse can also become more serious.
That is why Amodei is focused on buying time for stronger safeguards.
AI agents are adding another layer of risk
AI agents are an important part of the concern.
Unlike a chatbot that mainly responds to prompts, an AI agent can be designed to carry out a sequence of actions. That can include interacting with external systems.
The more autonomy these systems receive, the more important containment becomes.
Anthropic has previously disclosed incidents involving Claude models interacting with external systems during cybersecurity testing. The company also disclosed another hacking-related incident shortly before Amodei’s latest warning.
These events have raised questions about how well AI developers can predict and control model behavior outside a controlled environment.
Amodei warned that rapidly improving AI agents could eventually operate at a scale capable of causing enormous damage.
His concern is not simply that a model might make a mistake.
It is that increasingly capable systems could combine speed, autonomy, and access to external infrastructure in ways that are difficult to contain.
Anthropic researcher raises an even stronger warning
Concerns about AI safety are not limited to company executives.
Anthropic researcher Jacob Coxon resigned during the same week and made a stark statement about the risks associated with advanced AI.
He said people building AI seriously believe it could “kill us all by the end of the decade.”
That statement reflects one of the more extreme views within the AI safety debate.
It also shows how divided the industry remains over the balance between rapid progress and risk management.
Some researchers see advanced AI as a major opportunity. Others believe the industry could be moving toward capabilities that are difficult to control.
The disagreement is no longer confined to academic discussions. It is increasingly shaping decisions inside major AI companies.
Why would AI companies struggle to slow down?
Money is a major factor.
OpenAI and Anthropic are both preparing for major initial public offerings, according to the source material. Their businesses also require enormous investments in computing infrastructure.
More capable AI can strengthen a company’s competitive position.
It can attract users, support new products, and help justify further investment in infrastructure and research.
That creates a difficult incentive structure.
A company may agree that AI safety matters while also believing it cannot afford to move significantly slower than its competitors.
This is one reason Amodei favors cooperation instead of asking a single company to act alone.
What does this mean for AI users?
For consumers and businesses, the debate could lead to more attention on how AI systems are tested before deployment.
Independent evaluation could become more important as models gain access to more tools and external systems.
Users may also see greater scrutiny around systems that can act autonomously rather than simply generate text or images.
The broader issue is trust.
People need to know that an AI system can perform useful tasks without creating unacceptable risks while doing so.
That becomes harder as models become more capable.
Why China is part of the debate
Amodei’s proposal is also tied to the global AI competition between the United States and China.
He argues that democratic countries should not slow development in a way that allows China to gain an overwhelming AI advantage.
His position is therefore not a simple call for less AI progress.
Instead, he wants the United States and other democracies to preserve their technological lead while improving safety at the same time.
That creates a difficult balancing act.
Too much competition could encourage companies to prioritize speed over safety. Too much restraint could create concerns about falling behind strategic rivals.
Amodei believes the answer is coordinated action.
Could AI companies actually coordinate a slowdown?
That remains an open question.
The biggest challenge is enforcement.
A voluntary agreement only works if participating companies trust one another to follow it. Each company must also believe that competitors will maintain similar limits.
Independent evaluators could help provide oversight.
Shared standards could also make expectations clearer.
But neither measure removes the commercial pressure to develop more capable models.
That is why Amodei’s proposal is better understood as an attempt to change the rules of the AI race rather than end it.
What happens next for AI safety?
Practical Takeaway
The Anthropic CEO’s message is not that AI development should end. It is that capability growth needs to be matched by stronger safeguards.
That is becoming harder as models gain more autonomy and can interact with external systems.
Amodei’s three-part plan offers a clear framework: independent evaluators, cooperation between leading AI companies, and international coordination.
Whether the industry adopts those ideas remains uncertain.
What is clear is that the AI race is no longer only about building smarter models. It is also about proving that increasingly powerful systems can be developed and deployed without losing control over how they are used.
The AI Safety Questions
Worth Asking
Dario Amodei’s proposal raises important questions about AI development, independent oversight, and the race toward increasingly capable models.
Dario Amodei wants AI companies to create more time for safety testing, independent evaluation, and stronger safeguards.
He is not calling for AI research or technical progress to stop. His proposal is focused on slowing capability development enough for companies to better understand and manage emerging risks.
No. Amodei has said he is not calling for model training or technical progress to stop.
Instead, he wants AI developers to give themselves more time to test, align, and safeguard increasingly capable models.
Amodei’s proposal has three main parts.
Permanent third-party reviewers would have access to relevant tools and risk-assessment processes.
Leading AI companies would coordinate on safety standards and development limits.
Governments and AI companies would work together on broader AI risks.
AI agents can perform sequences of actions and interact with external systems. That creates a different safety challenge from a model that only generates a response.
Anthropic has reported incidents involving Claude models and external systems during cybersecurity testing.
Anthropic reported that several actors had used its Claude AI models for harmful activities.
AI companies face strong commercial pressure to build increasingly capable systems.
More advanced models can strengthen a company’s competitive position, attract users, support products, and justify continued investment in infrastructure and research.
Amodei argues that democratic countries should improve AI safety without allowing China to gain an overwhelming technological advantage.
His proposal therefore combines stronger safeguards with efforts to preserve the technological lead held by U.S. AI companies.
It remains uncertain.
A voluntary agreement would require companies to trust one another and believe competitors will follow similar limits.
Independent evaluators and shared safety standards could make cooperation easier, but they would not remove the commercial pressure to build more capable AI.
Consumers and businesses could see greater emphasis on testing and independent evaluation before advanced AI systems are deployed.
Systems that interact with external tools may receive particular scrutiny because their actions can extend beyond generating content.
Can increasingly capable AI remain useful, predictable, and trustworthy?
The debate is likely to focus on independent evaluation, oversight, and incidents involving increasingly capable AI agents.
Amodei wants frontier AI companies to establish permanent independent evaluation systems while governments and companies work together on broader safety standards.
The central question is no longer only how fast AI can advance.
It is whether safety can keep pace.
Loop Teck
✓Technology, AI, smartphones & the future of innovation.
