Advanced AI Could Outpace Human Control, Anthropic Warns

Artificial intelligence company  Anthropic⁠ has called for a coordinated global slowdown in the development of the world’s most advanced AI systems, warning that the technology may soon evolve beyond the capacity of existing institutions to regulate effectively.

 In a newly released report, the San Francisco-based company behind the Claude family of AI models argued that humanity should retain the ability to temporarily pause frontier AI development if technological progress begins to outpace safety research and governance structures.

According to Anthropic, a worldwide agreement involving major AI developers and governments would be required to make such a pause effective.

The company noted that if only one organisation halted development, competitors would likely continue advancing the technology, undermining the intended safety benefits.

Anthropic warned that the rapid acceleration of AI capabilities is creating conditions that could eventually lead to what researchers describe as “recursive self-improvement” a scenario in which AI systems become increasingly capable of improving themselves with limited human intervention.

 While acknowledging that such a development is not inevitable, the company argued that governments and institutions may not be adequately prepared if the technology advances faster than anticipated.

Anthropic compared the challenge to international nuclear arms control efforts, but noted that AI systems are considerably harder to monitor because research and training activities can be conducted more discreetly than traditional military programmes.

The proposal is likely to face resistance from technology companies and policymakers who believe slowing AI development could allow rival nations, particularly China, to gain a strategic advantage in the rapidly evolving technology race.

Nevertheless, the company insists that meaningful international coordination on AI safety remains essential as increasingly powerful systems emerge.