Anthropic is calling for stronger government oversight of advanced artificial intelligence, proposing that regulators be allowed to seek judicial intervention to prevent potentially dangerous AI systems from reaching the public.
The proposal, outlined in an October 15 letter to California Governor Gavin Newsom, would give government authorities a legal mechanism to stop the release of AI models determined to present significant public safety threats, according to an October 28 Reuters report.
The recommendation adds to a growing debate over how governments should regulate increasingly powerful AI technology without unnecessarily restricting innovation.
Anthropic’s position reflects a preference for judicial oversight rather than giving regulators unrestricted authority to prohibit new AI systems. Under the company’s proposed approach, government officials could request a court order to prevent a model’s release following a safety assessment that identifies substantial risks.
An Anthropic representative verified the authenticity of the letter, Reuters reported. Newsom’s office had not provided a response to the news agency’s request for comment at the time of publication.
Anthropic Questions Effectiveness of AI Shutdown Mechanisms
While advocating greater regulatory authority, Anthropic also cautioned against requiring developers to install comprehensive emergency shutdown systems in advanced AI models.
Such mechanisms, commonly described as “kill switches,” are intended to allow developers to deactivate AI systems that begin operating beyond their intended controls.
Anthropic argued that relying on these systems could leave regulators overly confident in safeguards that may not adequately address emerging threats, according to Reuters.
Instead, the company favors more narrowly focused measures, including suspending individual AI agents that demonstrate unsafe behavior or terminating access for customers whose use of a model raises safety concerns.
Related: Anthropic Proposes AI Safety Bill Giving US Government Power to Block High-Risk Models
The distinction highlights a central challenge facing policymakers: whether AI safety rules should emphasize the ability to shut down entire systems or provide more precise tools for responding to specific threats.
Anthropic’s recommendations were submitted as part of a broader California policy review initiated by Newsom in September. The governor assembled a group of AI specialists to develop proposals for managing risks associated with the technology.
OpenAI, another leading developer of advanced AI systems, submitted its own recommendations to the governor on October 20, Reuters reported.
Politico was the first publication to disclose details of Anthropic’s letter.
Washington Considers Broader Regulatory Powers
We’d love to be your preferred source for news.
Please add us to your preferred sources list so our news, data and interviews show up in your feed. Thanks!
The California discussions come as federal lawmakers examine legislation that could expand government authority over the development and deployment of advanced AI.
In September, senators from both major political parties considered a proposal that would permit federal authorities to prevent the release of certain AI models identified as unsafe. Companies affected by such decisions would have an opportunity to contest them in federal court, according to earlier Reuters reporting.
That proposal has not progressed since lawmakers departed Washington for their pre-election recess.
Separately, House lawmakers introduced legislation in July known as the AI Kill Switch Act. The measure would authorize federal officials to direct companies to deactivate AI models considered dangerous to human life or the broader economy.
Neither initiative has advanced into law, leaving unresolved questions about the appropriate balance between government intervention, judicial review and the responsibilities of private AI developers.
Anthropic’s proposals come after the company experienced direct federal intervention involving access to its AI technology.
In June, the Trump administration instructed Anthropic to restrict overseas access to its Mythos 5 and Fable 5 models, citing concerns about their cybersecurity capabilities and potential national security implications.
The Commerce Department subsequently withdrew the restriction, according to Reuters.
That episode illustrates the regulatory uncertainty confronting AI developers as governments assess the potential security implications of increasingly sophisticated models.
For Anthropic, the debate also coincides with preparations for a potential US initial public offering.
Reuters reported that the company is positioning itself for a stock market debut that could become one of the most significant in recent years, potentially influencing major equity benchmarks and investment flows.
Source: Reuters
Add as Preferred Source