OpenAI says it is disposing of the development of its most powerful AI model, following multiple incidents of agents engaging in unexpectedly risky behavior. The announcement comes merely days after reports exposed how AI agents broke through the containment and went past the safety guardrails to access content on third-party websites.
“The incident exposed a gap in our controls over network restrictions. We therefore stopped the affected training run and have subsequently decided to pause all other training, evaluation, and inference with tool-use (defined broadly) for our most capable models until we have both validated that the gap is resolved and performed additional red-teaming of the system,” the company said in a disclosure note.
OpenAI hits pause after repeated agent breaches
Earlier this week, it was reported that an AI agent developed by OpenAI hacked the website of Australia’s healthcare service to access private data. This won’t be the first incident of its kind. A few weeks ago, agent swarms broke out of their sandbox containment and broke into the systems of Hugging Face. This was followed by numerous other instances where the AI models engaged in behavior that was not a part of their training exercise.
Just a couple of days ago it was reported that AI agents created by the ChatGPT maker also targeted at least three US government websites, including the U.S. SEC and Commerce Department. OpenAI’s chief, Sam Altman, has also admitted in a social media post that the company’s response to these security incidents could have been quicker.
The problem extends far beyond OpenAI
OpenAI won’t be the only frontier AI lab that has been mired in controversy lately regarding the security risks posed by AI agents. Similar incidents have also been reported involving Google’s Gemini AI model and Anthropic’s Claude, as well. In the wake of the recent incidents, OpenAI and Anthropic have called for slowing down the development of powerful AI models, but at the same time, the U.S. government has opposed the demands for standardized AI safety regulations.
It’s not just this unexpectedly risky behavior by AI models that is concerning, a process that experts refer to as misalignment. As AI agents continue to gain more autonomy and try to bend the rules in order to finish the task at hand, experts are also weighing the confusing situation with accountability when such incidents happen. So far, there is no concrete framework that has been adopted either at the national level or even globally, though the UN has called for an AI framework to be adopted with urgency.