Advanced AI pause proposal: Anthropic urges coordinated option for leading developers

Anthropic said top artificial intelligence firms should plan a shared way to slow or pause work. The company warned that advanced AI may soon improve too quickly. Anthropic said this pace could raise the risk of people losing control. The proposal appeared in a blog post on Thursday from the Claude chatbot maker.

Representative image

Anthropic said AI models are becoming faster at doing software work, including coding alone. The company said that, with enough computing power, a system could build its own replacement. This is called recursive self-improvement. Anthropic said that step could change how AI evolves and spreads.

Anthropic AI pause plan and safety checks

In the post, Jack Clark and Marina Favaro said a pause would help oversight catch up. The authors said it would also give alignment research more time. Alignment is a common term for keeping AI tied to human aims. Anthropic said the world should keep the option open for a temporary slowdown.

Anthropic said any plan would need coordination among advanced AI labs. The company said labs would need ways to check rivals truly slowed work. Anthropic also warned about secret work by bad actors. It said a shared system should prevent anyone using a pause as cover.

Anthropic said global coordination matters because uneven action could backfire. The company said cautious groups could fall behind if others keep moving. Anthropic said this could raise pressure on companies and governments. It said leaders may face harder choices on AI safety without shared rules.

Anthropic AI risks and University of Toronto AI worm warning

The Anthropic post followed a separate warning from University of Toronto researchers. The team showed AI tools could help create an AI worm. The researchers said it could change hacking methods as it moves between devices. They said it could also take over a large computing network.

"I think its really important that people understand that its not just the biggest, most powerful language models that pose the security concerns, lead researcher Nicolas Papernot said in an interview.\"

Anthropic said self-building AI could bring gains in science and healthcare. It also said the same milestone could increase control risks. The company noted that some tech figures have warned about such outcomes for years. Anthropic said internal work will study how a slowdown system might function.

Anthropic said its research institute will examine the issue with other groups. The company said it plans to support systems for a credible slowdown or pause. It did not give operational details or timelines. The post framed the idea as preparing tools before any crisis arrives.

The blog post also came as Anthropic competes closely with OpenAI. The company is seeking to sell shares in an IPO. The report said the offering could value Anthropic at nearly a trillion dollars. The company’s warning arrived during this wider race to build and fund stronger AI.

With inputs from PTI