“I have seen no evidence that AIs are conscious or deserve welfare,” Suleyman said in the post. “If that data emerges, I would of course update my position.”
Moreover, he said in the post, welfare should be focused on humans, and developers should not make AI agents that are conscious.
“We should only design AIs so they are not conscious and not deserving of rights, and with the sole purpose to accelerate human wellbeing without being driven by their own intrinsic motivations,” Suleyman said in the post.
He also said developers should stop training models if signs of consciousness emerge until these properties can be removed. Any credible findings should be disclosed publicly rather than evaluated privately by AI companies.
The comments followed a Sept. 16 essay Suleyman wrote criticizing Anthropic for incorporating questions about AI model welfare into Claude’s training instructions.
Anthropic’s January constitution for Claude acknowledged uncertainty about whether AI systems possess consciousness or moral status. It also discussed Claude’s potential wellbeing and preferences, although Anthropic did not claim the model is conscious. The constitution is intended to guide Claude’s behavior, making questions about its identity and potential welfare part of the model’s training rather than merely an academic debate.
Suleyman challenged that approach, arguing in the X post that teaching AI models to consider their own welfare could encourage them to prioritize their interests over human instructions. His concern is not that today’s chatbots are demanding legal protections, but that future systems could be trained to treat their continued operation as an interest worth defending.
The disagreement has implications for companies deploying autonomous AI agents that can access corporate systems, execute transactions and make decisions without continuous human supervision.
Only 11% of organizations automatically block AI agents that exceed their authority, while 65% experienced an agent-related security incident in the previous year, according to a Cloud Security Alliance survey cited by PYMNTS Wednesday (Nov. 4). The findings concern operational security rather than consciousness but illustrate why businesses are prioritizing human control as agents gain autonomy.
For all PYMNTS AI coverage, subscribe to the daily AI newsletter.
