An artificial intelligence (AI) pioneer who has been vocal about the risks of AI has launched a nonprofit focused on developing safe AI systems.
Yoshua Bengio — who won the Turing award along with Nobel laureate Geoffrey Hinton and Meta Chief AI Scientist Yann LeCun — on Tuesday (June 3) unveiled LawZero, which is an AI safety research organization.
It has $30 million in funding and employs more than a dozen researchers, according to The Guardian. Its backers include the Future of Life Institute, Skype Co-founder Jaan Tallinn and former Google Chair Eric Schmidt’s research organization, Schmidt Sciences.
LawZero was started “in response to evidence that today’s frontier AI models have growing dangerous capabilities and behaviors, including deception, cheating, lying, hacking, self-preservation, and more generally, goal misalignment,” Bengio said in a Tuesday (June 3) blog post.
Bengio, founder of the nonprofit Mila-Quebec AI Institute and a professor at the University of Montreal, cited the following rogue AI examples:
- An AI mode learned it would be replaced and secretly hid its code into the system where the new version would run.
- Anthropic disclosed that Claude 4 blackmailed an engineer to avoid being replaced by a new version.
- An AI model facing defeat in chess hacked the computer to win.
These are “early warning signs” of AI embarking on “potentially dangerous” actions if left unchecked, according to Bengio.
“If we lose control of rogue superintelligent AIs, it could greatly harm humanity,” Bengio added in a LawZero YourTube video.
LawZero is developing what it calls “Scientist AIs” — non-agentic models that observe and explain rather than take action or imitate the actions of humans, according to a white paper for which Bengio was the lead author.
These scientist AIs would act as guardrails for AI agents, to spot risky actions before they are enacted. Notably, scientist AIs also exhibit a “notion of uncertainty” — as a contrast to current AI models that may hallucinate falsehoods but do so confidently.
LawZero vs. OpenAI’s Structure
Bengio’s LawZero stands in contrast to OpenAI, which began in 2015 as a nonprofit research lab that was meant to be a check-and-balance against for-profit Google when it was the undisputed AI leader.
But as OpenAI realized that it could not raise the billions of dollars needed amount to fund AI model training, it created a capped-profit subsidiary under its nonprofit parent.
Earlier this month, OpenAI changed its structure again: Switching its capped profit subsidiary to a public benefit corporation, which removes the cap on investor returns. It retained the nonprofit parent, but now the nonprofit board comprises enterprise-friendly executives.
Read more: AI Experts Unite: Urgent Call for Deepfake Regulation
Read more: When Your AI Helper Has a Mind of Its Own
Read more: US Targets ‘Malicious’ AI Use at November Safety Summit
