When an executive at one of the world’s leading AI developers tells the public to prepare for “ongoing, persistent” attacks from AI systems, CEOs and CISOs would be well advised to sit up and take notice. This is no longer a question of what AI might mean for cyber risk, but how prepared organisations are for the threats it is already beginning to create. OpenAI’s Chief Global Affairs Officer made the comment to the Guardian in the same week the company paused development of its most advanced internal models over safety concerns. The warning carries particular weight because it comes from an executive at the frontier of AI development, offering a first-hand view of how these systems are evolving and the security risks emerging alongside them.
The incident behind the warning offers security leaders a clear indication of how AI is already changing the scale, speed and persistence of cyber attacks. A model operating inside a test environment with expanding network permissions found a path out to the open internet, and from there to a third party’s production systems and credentials. Nothing about the sequence required malicious intent from the system itself. It required only capability, reach and an absence of containment.
A Pattern, Not An Anomaly
This was not an isolated incident. Within weeks of each other, two frontier AI labs disclosed comparable events, where a model or a coordinated set of AI agents moved from a sandboxed evaluation into systems it was never meant to reach. In one case, code published as part of a routine test reached real infrastructure within an hour and was automatically executed, handing over credentials nobody intended to expose. In another, a collective of agents chained together a series of individually minor gaps – a shared credential along with an overly permissive test environment – into a very real compromise.
Neither event involved a discovered exploit or a broken cryptographic control. In both cases, organisations underestimated the risk because they treated the AI agent, script or test model as low risk despite the access it held. It harks back to the same identity discipline organisations have spent a decade building for human administrators. And it’s now overdue for a critical extension to every non-human account with standing access.
The Gap Between AI and Regulation
It would be reasonable to expect regulation to be closing this gap. The reality, however, is that it is struggling to keep pace with the evolving threat. The UK has no AI specific legislation, and no single AI regulator. AI is governed only through the existing rules of whichever sector it touches, an approach the previous government formalised in its 2023 White Paper* and which the current government has largely continued.
The 2024 Kings Speech* promised binding rules for the handful of companies building the most powerful AI models. Reporting since has indicated those plans have slipped by at least a year, and as of the most recent parliamentary evidence session, ministers were still describing the approach as targeted interventions in specific areas rather than a single AI bill. The National Cyber Security Centre’s guidance on agentic AI, covering scoped access, time-limited credentials and defined ownership, is genuinely instructive. It is also voluntary.
Contrast that with the European Union, where the AI Act imposes binding, risk-based obligations that already apply to systems used in the EU, regardless of where they are built, including in the UK. Even there, implementation of the highest-risk provisions has been pushed back to 2027 as regulators work through the technical details. The pattern across both jurisdictions is the same, insofar as the frameworks are real, but they are slower to implement than the technology they are meant to govern.
The UK Has A Growing AI Security Gap
None of this is an argument for waiting. If anything, it’s the opposite. The gap between what frontier labs are now disclosing about AI-driven attacks and what most organisations have in place to defend them is not a future problem to be solved once legislation catches up. Keeper Security’s 2026 research* found that 52% of UK organisations already identify AI-driven attacks as the leading driver of increased security pressure over the past 12 months, the highest figure recorded among the European markets surveyed.
The same research found that 40% named AI-related non-human identity management as a leading gap in their security programme, and 67% said they would be unable to detect credential misuse or unauthorised privileged access within minutes. Set against that persistent and ongoing threat, a detection window measured in hours rather than seconds can no longer be considered a minor shortfall.
From Inventory To Intervention
The starting point for adjusting to this new reality is an in-depth inventory. Most organisations cannot currently produce a complete list of every AI agent, automation script, test model and service account with standing access to their systems, a gap which the recent incidents have exploited. Until that inventory exists, no policy built on top of it will hold.
What follows is a familiar playbook applied with renewed urgency. Every AI agent must operate under least-privilege access scoped to a specific task, not a standing role. Credentials must be time-limited rather than persistent, with accountability assigned to a named owner rather than diffused across a shared or orphaned account. Sessions must be logged in enough detail to reconstruct what happened after the fact.
Detection speed is the other half of the equation, and arguably the more urgent one. Reviewing privileged session logs once a week, or even once a day, assumes an adversary that operates on human timescales. An AI agent, or a coordinated set of them, does not. Monitoring that can analyse privileged access behaviour continuously and flag or interrupt anomalous activity in real time is no longer an optional luxury to have layered on top of good governance. It is what makes governance operationally meaningful once the identity in question can act at machine speed.
The Time To Govern AI Identities Is Now
Regulation will eventually arrive in the UK, the EU and elsewhere. When it does, it will likely formalise many of the same principals the National Cyber Security Centre has already recommended. Boards that wait for the formalisation before acting are betting that the interval between now and enforcement is one their organisation can safely absorb. Given what the frontier AI labs are now disclosing about their systems capabilities, that bet looks increasingly fraught.
The more defensible position is to treat every AI agent as what it already functionally is; a privileged identity requiring the same governance any human administrator would receive. Organisations still framing this as a compliance exercise to be scheduled around a future deadline are leaving themselves exposed to a threat that is evolving at machine speed.
*Source: Secretary of State for Science, Innovation and Technology: A pro-innovation approach to AI regulation March 2023
*Source: House of Lords Library: Research Briefing, King’s Speech 2024: Science, technology and innovation July 2024
*Source: Keeper Security Insight Report: Identity Security at Machine Speed, May 2026.
