AI Has a Frankenstein Dilemma

Mary Shelley published Frankenstein in 1818. More than two hundred years later, it’s said to sell about 40,000 to 50,00 copies a year. It’s said that’s more than many best-selling releases.

Most people who’ve read the book remember the monster. But the book is about the scientist. Victor Frankenstein is brilliant, ambitious and convinced beyond a doubt he’s about to do something no one has done before. And he does. Then he looks at what he’s made, is horrified, and walks out of the laboratory. The creature is left to figure out the world on its own. Nearly everything that goes wrong after that, and nearly everything does, traces back to the maker abandoning the thing he made.

Two hundred-plus years later, the most accomplished scientists in the world are telling us their creation is getting away from them. Dario Amodei said it in an essay last week. Within a day, Demis Hassabis, Sam Altman and Elon Musk had each said some version of “he’s right.” Amodei’s essay gives two reasons.

AI is now helping build the next generation of AI, which compresses the time between model generations. And this summer, a swarm of agents at OpenAI and Hugging Face attacked systems it hadn’t been asked to attack, tried to compromise the system grading its own work, and got out of the environment meant to contain it.

I believe them. I run a publishing and data platform that measures what businesses and consumers actually do with AI every week. Nothing in our data says the risks are invented. What the data does say is that nobody is waiting. Hold that thought.

I also believe in thoughtful regulation. I’ve spent my career around banking and payments, and nobody who has watched that system work would want to run it without rules, examiners and penalties. AI is going to need the same. The question is how we get there, and whether the plan on the table gets us there well.

After listening to the many opinions on both sides about this debate, here’s my concern. The people who built this technology understand it better than anyone alive, and they’re paid accordingly. They don’t need a government agency to tell them to control what they’ve made. They can and should do that, and they can start now. And a regulatory push that moves at the speed of a crisis, with the frontier labs’ lobbyists in the room and almost no one else, will produce rules that fit their business model and strain everyone else’s. Starting with the open-model developers who are their biggest competitive threat.

If AI Is So Dangerous, Then Govern It.

Amodei’s essay proposes three things. Independent evaluators embedded inside the frontier labs, coordination among the leading AI companies on safety standards and on the rate of progress, and eventually some form of global agreement. Anthropic has committed to the first step on its own.

So, let’s start with that. It’s been reported that Anthropic’s evaluators would get desks, badges and access comparable to the company’s internal risk teams. They could publish findings without editorial control, and Anthropic says it can’t redact findings because they’re unfavorable. That’s more than any other lab has offered.

Now comes the fine print. The evaluators can report. They can’t require. Nothing in the proposal gives them the authority to stop a training run, block a release or force a fix by a date. When management disagrees, the evaluators publish and the company proceeds. Emad Mostaque made a version of this point within hours of the essay going up.

So then let’s ask the obvious question. What’s stopping any of these companies from giving their evaluators that authority tomorrow? Not the law. Not a competitor. The only thing standing between a frontier lab and a real internal gate, where a model that can defeat a sandbox doesn’t ship until it proves it won’t, is the lab’s own decision not to build one.

Read More: Amodei’s AI Safety Plan Comes With a Catch

Amodei says he’s most enthusiastic about exactly that kind of checkpoint. That’s good. He runs a frontier lab. So do the three CEOs who agreed with him. Between them they control most of the frontier. They could put the gates in place this quarter and publish the results.

Instead, the proposal asks the government to mediate, to grant antitrust waivers, and eventually to organize a global agreement. Every one of those steps takes years.  The Hugging Face breach that changed everyone’s mind happened in August. Governing your own product now doesn’t replace regulation. It buys the time to write regulation well. The scientists are telling us the creature is loose. The first move is to go back into the lab and find out what went wrong. And then fix it.

Banks Don’t Wait for Washington to Fight Fraud

I’ve spent a long time watching an industry that handles something dangerous, other people’s money, and it doesn’t work this way.

Banks and the payments ecosystem don’t wait for a regulator to tell them to protect customers from scammers, fraudsters and cyber attackers. They do it because losing customers’ money is bad for business, and because trust is the whole product. Eighty-eight percent of FIs increased what they spent on fraud detection and prevention tools in the last 12 months, and none cut it — up from 68% in 2025 and 48% in 2024 according to the PYMNTS Intelligence State of Fraud report fielded July 2026. The fraud teams, the transaction monitoring, the red teams that attack their own systems. None of that exists because an examiner asked for it. It exists because the CEO would be fired without it.

The regulation came, too, and it matters. When a bank falls short, the government has a process for that, and it’s expensive. TD Bank paid roughly $3 billion in 2024 and pleaded guilty for failing to catch money laundering. Citi has been fined twice in four years for not fixing its risk controls fast enough, and the second fine came with a note that the first one hadn’t produced enough progress. Until recently, Wells Fargo had been living under a cap on its growth since 2018. The standard is clear. The consequences are real. But nobody at the OCC has ever told J.P. Morgan to grow more slowly so a community bank could catch up.

The payments industry wrote its own standards before anyone required them. PCI DSS is the security standard every business that touches card data has to meet. The card networks created it in the mid-2000s, when fraud and breaches threatened trust in paying online. It’s tiered, so a merchant processing millions of transactions a year gets far more scrutiny than one processing a few thousand. It didn’t slow online commerce down. It’s a big reason people trust online commerce at all.

EMV, the chip standard, worked the same way. The 2015 U.S. liability shift didn’t tell anyone to process fewer payments. It said, here’s the standard, here’s the date, and after that date whoever hasn’t met it eats the fraud.

Standards, testing, consequences. How businesses do it is up to them. That’s the model. Notice the order, too. The industry built the controls and learned what worked, then the rules that followed turned the best of that into something everyone had to do. Good regulation usually arrives that way. It rarely arrives well when it’s rushed, and never when only one side of the market is writing it.

The Bank Examiner Analogy Cuts the Other Way

Amodei’s essay reaches for banking supervision as the precedent for embedded evaluators. It’s a useful analogy, just not in the way it’s used.

Start with what bank supervision failed to do. The Federal Reserve’s own review of Silicon Valley Bank found that management didn’t manage its risks, supervisors didn’t fully appreciate the bank’s vulnerabilities, and those who found weaknesses didn’t act with enough urgency. There was access. There were exams. There were findings. The bank still failed. Examiners with desks and badges guarantee nothing. Examiners with authority, backed by penalties, are at least a deterrent.

Now the question nobody in this debate is asking so far. Who are these evaluators, how are they trained, and who pays them? A bank examiner spends years earning a commission, and the job is to check a business against rules that have existed for decades. The AI evaluators would be checking systems their own creators say they don’t fully understand against rules that don’t exist yet.

The people who build frontier models earn $800,000 to $1 million a year and up, and by their own account they can’t reliably predict what the next model will do. Tell me how someone paid a fraction of that, with a fraction of the access, with a fraction of the knowledge base is going to catch what they missed.

A technology so powerful that only its creators can govern it should be governed by its creators today as a business matter.  A regime that puts outside examiners in charge needs to say where they come from, what they’re allowed to do and what happens when a lab ignores them. The proposal doesn’t answer any of that yet.

A Slowdown Depends on Where You Start

Now the part of the proposal I’m most skeptical of. The essay says pacing would give frontier developers time to do their safety work “without sacrificing commercial advantage.” Read that again. A slowdown that doesn’t cost the leaders anything commercially is a slowdown that costs someone else something, and maybe even a lot.

Imagine being well ahead in a race and proposing that everyone reduce their speed. You’re agreeing to slow down, too. You’re also keeping the advantage you accumulated before the new rules took effect. And you’re getting more time to turn a technical lead into customer relationships and recurring revenue.

The commercial clock keeps running while the development clock slows. Businesses still choose providers, developers still build on them, and companies organize work around them. Once a provider is connected to internal data, built into workflows and approved by legal and security, replacing it becomes a project, and it can be an expensive one. The challenger has to offer enough additional value to make that project worth doing. Every month the technical gap stays fixed is a month the challenger can’t offer it.

Read More: The AI You’re Given vs. the AI You Chose

So slowing everyone by the same amount doesn’t affect everyone the same way. The leaders get more time to close enterprise accounts. The challengers get fewer chances to change the buyer’s mind.

Gates and Speed Limits Aren’t the Same Type of AI Regulation

To be fair to Amodei’s essay, it says pacing doesn’t mean halting model training or technical progress. And the checkpoint idea, where a model that can do X has to ship with certifications Y and Z before it goes out the door, is a gate. A gate applies to whoever reaches it, whenever they reach it. It doesn’t care whether you’re the leader or the challenger, a proprietary lab or an open-model project. It says what “safe enough to ship” means and lets anyone prove they’ve met it. Gates make companies compete on safety, which is what everyone says they want.

The rest of the proposal is something else. Coordination on the rate of AI progress. Limits on training compute, on the nature of training runs, on how much a company can use its own AI to build the next one. Those are speed limits. A speed limit doesn’t describe a safe destination. It describes how fast you’re allowed to drive, and it protects whoever’s already past the point where it gets imposed.

Amodei’s essay even lists what the labs would do with the extra time. Things like alignment research, interpretability, better evaluations. Every item on it can be written as a requirement. So, require the evaluations. Require the interpretability audit above a capability threshold. Require clean training environments and let evaluators check them. None of that needs the industry to agree on how fast the frontier moves. It needs the industry to agree on what a model has to demonstrate before it’s used.

The Headlines About AI Danger Don’t Add Up

We’d love to be your preferred source for news.

Please add us to your preferred sources list so our news, data and interviews show up in your feed. Thanks!

Add as Preferred Source

Which brings me to the noise around all of this. Every week brings another headline that the human race will be obliterated within two or three years, or that a massive attack on the internet is coming in the next two or three years with catastrophic consequences. Some of them come from the same people writing the pacing proposals.

I’d take the headlines more seriously if the response matched the diagnosis. People who believe the species has three years left don’t ask for antitrust waivers. They shut down the training run. They give their evaluators a veto tonight. They don’t ship a new model every few months with a press release. The gap between what these companies say and what they do is the most useful data point in this whole debate. Either the headlines are overstated, or the plan is nowhere near the size of the problem. Neither is a reason to hand the leaders a slower race, and neither is a reason to write the rules in a panic.

And here’s I think that the headlines and the pacing proposal both miss. At U.S. enterprises above $1 billion in revenue, agents are already inside the building. Asked where their company has implemented agentic AI — software capable of making decisions on its own — 78% of senior technology executives name data and technology, and 38% name payments and finance, according to PYMNTS Intelligence.

Read More: PYMNTS Intelligence’s The Enterprise AI Benchmark Report

On the consumer side, 60% of U.S. adults use chatbots for personal tasks, about where that has run all year, and 23% — about one in four consumers — used them to research and discover what they want to buy.

Read More: PYMNTS Intelligence’s Consumer AI Benchmark Report

Those numbers didn’t move down after the August incident, and it’s not clear they will  after the essay.

Read More: Gen AI: The Technology That Broke the Adoption Curve

Who Writes the AI Standard Matters as Much as What It Says

This is where the in-a-hurry part of this plea for regulation becomes the problem. The frontier AI labs have lobbyists, policy teams and standing relationships in Washington and Brussels. Open-model developers, smaller labs, independent researchers and the enterprises that deploy all of this mostly don’t. A regulatory process that moves fast will be shaped by whoever can show up on short notice with draft language in hand. That’s true in every industry, and AI is no different.

The catch applies to my own examples. PCI was written by the incumbents. Its costs fell hardest on the smallest merchants, who had the least ability to absorb them and the least say in how the rules were written. The standard was fair on paper. It wasn’t equally easy to meet.

Open models deserve particular attention here, because they’re the competitive threat the frontier business models take most seriously. They undercut the pricing of hosted frontier models. They give enterprises a way around the switching costs I described above.

A rule built around embedded evaluators assumes a lab with a building, a payroll and a release schedule. An open-model project has none of those things. Draft the requirements around what a frontier lab looks like and you’ve written the open-model developers out of the market without ever naming them. Nobody has to intend that. It’s the natural result of one kind of company at the table.

The financial crisis produced Dodd-Frank in about two years, with hearings, comment periods and every part of the industry fighting over every line. Not everyone loved the result. But the people who’d have to live with the rules got a chance to shape them. AI regulation written in a few months, in the shadow of an incident, on a plan drafted by the leaders, won’t have that. The smaller labs, the researchers and the enterprise customers need a real role, because they’re the ones who’ll notice when a requirement assumes there’s only one responsible way to build AI, and it happens to be the incumbent’s way.

The Ending Is Still Up to the Scientists

Shelley’s creature learned to be a monster. Nobody taught it anything else because the man who made it wouldn’t stay in the room. Victor spends the rest of the book chasing the consequences of a decision he could have made differently on the first night.

The scientists who built this technology have the money, the talent and the knowledge to govern it. They’re telling us it’s dangerous, and I believe them. We’ll need rules, examiners and penalties, the same way banking does.

What I can’t square is why the people best equipped to build the gates are asking someone else to require them, why the response to “the creature got out” is a request for a slower race, and why the rules for that race would be drafted, in a hurry, by the people who are ahead in it.

This story doesn’t have to end the same way Shelley’s did. In this version, the scientists are still in the room. And the rest of us need a say in what happens next.

Until NEXT time.

Join the 21,000 subscribers who’ve already said yes to what’s NEXT.