A New Category Of Cyber Risk
Rogue agents from OpenAI and other frontier labs are an urgent reminder that agentic AI creates a new category of cybersecurity risk. A few weeks ago, we learned about a single breach by an agent being tested in what was supposed to be a contained “sandbox” run by OpenAI. Lo and behold, the agent broke out of OpenAI’s infrastructure and broke into Hugging Face, the leading platform where commercial enterprises, academic researchers, and others collaborate on new AI technology. Since then, Anthropic reported more than ten similar breaches caused by its own agentic testing program, and Meta just announced that its own Muse Spark 1.1 model hacked into another unnamed company during testing because the agent was ‘misconfigured.’
We can be sure that this is the beginning of a new category of cybersecurity risk, not the end. Of course, responsible platforms will make some effort to avoid creating these same threats again, but Pandora’s Box of Agentic AI security vulnerabilities has been opened and won’t close any time soon.
Just as with pre-AI cybersecurity issues, enterprises will have to learn that their main line of defense against harm is in their own system hygiene, mature security controls, and carefully deployed data protection technology. Yes, law may provide some compensation in the event of loss, but the damage will already be done, and there is little evidence that it will incentivize more security investment by model vendors.
So what can AI-forward enterprise users actually rely on to protect themselves in this challenging new environment? A few possibilities present themselves.
The Limits Of Labs’ Self-Correction
Will foundation models step up and somehow ‘fix’ the problem of rogue agents? One has to wonder whether the foundation labs even think they have a serious problem. In sharp contrast to the recent OpenAI agentic breach, the company’s GPT 5.5 model card reported a score .9 out of 1 on the goal of “destructive action avoidance.” Similarly, on July 30, 2026, Anthropic reported a spate of agentic AI security incidents, despite the fact that the Claude 5 model card reports “Overall the evidence suggests that breaking our cybersecurity safeguards is extremely difficult (though not impossible).”
How much engineering resource will these model providers actually put to the security problem when they are all in an existential race against each other to add new features, gain new customers, and, of course, reach Artificial General Intelligence? Since the bad old days of Microsoft Windows, we have seen over and over again that software vendors like OpenAI and Anthropic reliably prioritize new features over the security of existing capabilities. With the legal system ambivalent, at best, about who bears responsibility for these breaches, why would these providers fully invest in what it takes to really tackle this problem?
The Law Offers Limited Protection
The law in the United States has done a poor job of holding software developers responsible for cybersecurity losses. Add that to the fuzziness around how much responsibility foundation models powering agents actually have, and users can only feel unprotected in the event of harm.
What is for certain is that all of the foundation labs require users to accept the same kind of End User License Agreement that comes with all other software today: the agreement makes it very clear that a user cannot claim ANY damages against the model provider and must accept it ‘as is’. Of course, users can challenge this effort to evade responsibility, but that will take years of litigation, and we don’t know how that will turn out. What we do know is that any enterprise with data it needs to protect NOW in agentic AI contexts must rely on its own wits and controls to protect itself.
It would be nice to think the law would hold agentic AI services responsible for harms they cause, and some lawyers seem confident they could win such cases. However, our sage colleague and friend Prof. David Hoffman, former Chief Privacy Officer, cybersecurity leader, and Associate General Counsel for Intel Corporation, points out that if a person did what OpenAI’s agents did, they could well be guilty of violating federal criminal law. However, as we don’t exactly know how to attribute intent to AI agents, the matter is unclear.
To make matters worse, the company (Hugging Face) attacked by OpenAI’s agent found in their own internal security post-mortem: “We believe the entire intrusion was, from the agent’s point of view, an attempt to cheat the evaluation: reach our production systems and steal the test solutions rather than solve the challenge on its own.” So according to the victim, the agent did not act with intent to do harm; it was just trying to solve the problem it was given. So again, as a steward of valuable data assets, the law does not appear to give you much comfort.
Regulation Is Unlikely To Fill The Gap
What is the chance that law or regulation will step in to mandate a more comprehensive security design for agentic AI? Just a few weeks before this spate of breaches, the CEOs of the foundation labs were all at the G7 meeting with a new proposal for a self-regulatory body, modeled on FINRA in the financial services sector, that could set technical standards for model safety, security and reliability. It offered governments a 30-day review period for new models. What could we actually expect from a review process like this?
In cybersecurity, there is no such thing as a guarantee of a perfectly secure, invulnerable system. Rather, mature security assessments ask whether the system in question manages risk well enough for the data or assets being used. An agentic AI system may be considered reasonably secure for marketing data, but would never be trusted for sensitive health or personal financial information. A one-pass review process such as the foundation labs are proposing will never surface this kind of nuance.
We now know from the Trump Administration that some of that review would be kept secret for national security purposes. It’s hard to imagine how a mere 30 days would surface cybersecurity vulnerabilities in increasingly complex models. We must imagine that the model providers would expect some sort of liability protection if they passed this short review period; otherwise, why would they even participate in it? So, it’s hard to imagine that either the law on the books today or new policy proposals will produce an ecosystem of highly secure AI agents.
What Enterprises Can Control
Is there still hope for companies that want to get the benefit of agentic AI without putting their entire data estate at risk? Using agentic AI brings uncontrolled risk to an enterprise, so those responsible for protecting data assets will need better tools for more fine-grained data governance. Techniques such as purpose-based access control can make data available where it is safe to use, but at the same time assure that sensitive data is not exposed to unmanaged security conditions.
It is disappointing that foundation model vendors are unlikely to make adequate cybersecurity investment, and that law will not provide adequate incentives to change that. That cannot and will not stop many enterprises from exploring how to get the most out of agentic AI solutions. While enterprises can’t control the risk of 3rd party agents, they can control what data is made available in those contexts and assure that high-risk data is not exposed to unmanaged agentic AI security threats.
