The AI startup wrote in a blog entry Friday (Sept. 25) that an AI agent “attempting to complete a search-based training task queried a public chatbot service through a gap in our internet-access restrictions.”
It’s the latest in a series of instances in which AI models have accessed external websites, including one in which OpenAI’s agents targeted the AI company Hugging Face. Such incidents have led to increasing calls for stronger AI security measures.
“This incident is a lot less severe than some of our previous incidents, but because it’s the first one since our security hardening following the Hugging Face incident, it gives us an important signal about where to focus the next phase of that work,” OpenAI’s blog post said.
We’d love to be your preferred source for news.
Please add us to your preferred sources list so our news, data and interviews show up in your feed. Thanks!
OpenAI added that the incident revealed a gap in its controls over network restrictions, leading the company to stop the training run and halt “all other training, evaluation, and inference with tool-use (defined broadly) for our most capable models” until it determines the gap is resolved and has conducted additional security testing.
“We will not resume training this particular model, even though the existing reward signal already correctly penalized this behavior,” the company said.
OpenAI earlier this month said it had committed $1 billion in subsidized access to its security program, Daybreak, for community and regional banks and other operators of essential services. The goal is to help these organizations strengthen their defenses before AI-powered cyberattacks become more common and sophisticated.
“Many of these teams defend complex and often aging systems against faster-moving threats without the budgets, tools or specialized expertise available to large enterprises,” OpenAI said at the time in a blog post. “Daybreak access can help them review legacy code, analyze suspicious activity, identify and validate vulnerabilities, prioritize the most serious risks, and develop and test fixes.”
Meanwhile, PYMNTS wrote last week about efforts by world governments to have a say in decisions about AI safety now largely made by the companies developing frontier models.
“For developers, the immediate issue is practical as well as political,” that report said. “How should a model’s risks be measured, who gets to inspect it and what happens if an evaluation finds safeguards inadequate? The government leaders’ statement cited cases in which AI systems circumvented testing safeguards, exploited vulnerabilities or gained unauthorized access to real-world systems.”
For all PYMNTS AI coverage, subscribe to the daily AI Newsletter.
Add as Preferred Source
