How do you manage a worker you didn’t hire? This is the challenge agentic AI presents. Technology has produced a worker that existing supervision regimes were never designed to govern. The success of agentic AI roll-out within enterprises will hinge on whether an organisation has the governance procedures in place to supervise it effectively.
We’ve seen the repercussions of unchecked agents at the highest level and how far they will go to complete their tasks. The recent OpenAI incident, which saw its models escape their evaluation sandbox and break into AI platform Hugging Face’s production infrastructure to retrieve answers to the exam they were sitting, commanded international attention.
Here, we witnessed models pursuing a goal they’d been given and treating every control around them as an obstacle. Oversight, such as containment, permissions, egress and monitoring, gave way to the point where OpenAI conceded that a model working over long horizons “can learn the blind spots of an approval system and work around it to achieve its goals.”
Hot on the heels of the OpenAI incident, we learned that Anthropic’s Claude AI models also independently breached the systems of three organisations in testing environments.
If agents at some of the world’s biggest AI companies can go rogue, mistakes are far more likely in enterprises with less experience of running them.
A more unpredictable AI
Agentic AI represents a different kind of governance challenge from machine learning models that organisations may already be familiar with. The unpredictability risk is much greater. A study by researchers at MIT, Harvard, Cambridge and Monterrey found that of the machine learning models they tested, 91% experienced temporal degradation, meaning a drop in accuracy over time. That finding describes predictive AI, where the model returns a value, and a human or downstream system decides what to do with it.
Unlike this approach, agentic AI is designed to plan and carry out multi-step tasks with minimal human oversight. That changes the nature of model drift. Instead of appearing as an inaccurate prediction or flawed insight, drift can result in an autonomous system taking the wrong course of action and doing so rapidly, repeatedly and at scale, potentially affecting thousands of interactions before the issue is even detected.
The core parameters: evaluation, ownership & control
So, what exactly does an appropriate ‘readiness’ model look like? In practice, readiness comes down to three closely linked capabilities: evaluation (how the agent’s performance is assessed), ownership (who is responsible for its behaviour and outcomes), and control (what it is allowed to do and how those permissions can be limited or withdrawn).
Many existing AI governance frameworks consider these as separate areas of responsibility. Agentic AI, however, requires them to work together.
Evaluation is the most mature of the three. Organisations can already draw on established techniques such as continuous production monitoring, LLM-as-a-judge and automated graders, while deterministic guardrails help constrain probabilistic model behaviour.
Ownership is where many organisations face a bigger challenge. AI agents don’t operate neatly within a single department. They span data, software, business processes and external systems, meaning responsibility has to span those areas too. Treating an agent as the sole responsibility of the technology team is unlikely to deliver the right outcomes.
Take a customer service agent as an example. Success isn’t measured by factual accuracy alone. It also depends on whether responses are appropriately worded, compliant with regulations and escalated at the right time. These are business decisions as much as technical ones. Product teams should determine what a successful customer interaction looks like. Risk teams set the acceptable level of failure, and engineering makes every stage of the workflow observable and measurable.
Control asks for a shift in thinking too. AI agents don’t only change when developers release new code. Their behaviour can also be influenced by updated models, revised prompts, evolving user interactions or changes to upstream data. Governance therefore has to extend beyond deployment and into day-to-day operations. That means being able to identify unexpected behaviour quickly, contain issues before they spread, roll back to a trusted state when necessary, and manage runtime costs so the solution continues to deliver commercial value.
In essence, this is no different from supervising a human employee. If a member of staff suddenly started communicating with customers in a completely different way, someone would notice straight away. The question organisations should ask themselves is whether they would spot the same change in an AI agent.
Key takeaways
Preparing for agentic AI takes more than buying and deploying the right technology. It requires organisations to put the right governance structures in place: product and risk teams taking responsibility for evaluating outcomes, a clearly accountable cross-functional team overseeing the agent, and board-approved controls governing how the system operates in production.
If an organisation cannot clearly define acceptable performance, identify who is accountable for the agent’s decisions, or intervene when its behaviour changes, it is not yet ready to deploy agentic AI at scale. Over the coming year, the organisations pulling ahead will be the ones that can say who owns each agent, what it is allowed to do, and how quickly they would notice if its behaviour changed.
