An AI governance dashboard can be entirely green the day before something goes wrong.
No major incidents. No breached risk limits. Required assessments completed. Controls operating within stated thresholds.
Yet elsewhere in the organisation, people may be overriding the same AI system more often. Guardrails may be blocking an increasing number of actions. Policy exceptions may be accumulating. An AI-enabled workflow may be behaving differently from its established pattern.
None of those signals is necessarily an incident. Together, they may be telling a very different story.
As AI becomes more embedded in enterprise workflows, governance needs to evolve from measuring whether failure has occurred toward detecting whether the conditions for failure are beginning to form.
The absence of AI incidents is not evidence of effective AI governance.
Monitoring is necessary. Risk sensing is different.
Post-deployment monitoring is already becoming a core expectation of responsible AI governance.
NIST’s 2026 report on deployed AI monitoring notes that real-world operation can introduce unforeseen outputs, dynamic conditions and unexpected consequences that may not appear during pre-deployment testing. NIST also describes monitoring practices as fragmented and still maturing across the AI ecosystem.
Europe is moving in the same direction. The EU AI Act’s post-market monitoring provisions require monitoring for relevant high-risk AI systems, while its human-oversight provisions expect organisations to enable people to detect anomalies, dysfunctions and unexpected performance, and to intervene where appropriate.
But monitoring and risk sensing are not the same thing.
Monitoring asks: What is happening?
Risk sensing asks: What patterns are emerging, and what might they be telling us before a material event occurs?
That distinction matters more as AI becomes increasingly embedded in decision-making, customer interactions and automated workflows.
The danger of governing through incidents
Incident metrics are attractive because they are concrete.
An AI system caused a customer-impacting error. A privacy control failed. An inappropriate output reached a user. An automated action created financial or operational impact.
These events can be counted, investigated and reported. But they are lagging indicators.
By the time an organisation records a material AI incident, earlier signals may already have existed.
A reviewer may have repeatedly corrected the system. A control may have blocked similar behaviour several times. A team may have requested repeated exceptions. An escalation pattern may have shifted without crossing a formal threshold.
The governance challenge is recognising when individually weak signals are beginning to form a meaningful risk pattern.
An AI Governance Signal Stack
A practical way to think about this is through an AI Governance Signal Stack.
The stack does not assume every weak signal will become an incident. It gives organisations multiple layers through which deterioration in AI behaviour, controls or governance can become visible.
1. Behavioural anomalies
The earliest signals may be subtle changes in behaviour.
An AI system may begin producing a different distribution of outputs. A workflow may invoke tools differently from its historical pattern. Escalation behaviour may change after a model, prompt, workflow or upstream data source is updated.
None of this automatically means the system is unsafe. But deviation from an established pattern can provide an early reason to investigate before a defined control fails.
2. Governance friction
The next layer is what people do around the AI system.
Are users increasingly overriding its recommendations? Are teams repeatedly seeking exceptions to the same policy? Are human approvers becoming unusually fast, or unusually reluctant, to challenge AI-generated decisions?
Human interaction with AI is itself a source of risk intelligence.
A rising override rate may indicate deteriorating AI performance. An exceptionally low override rate in a high-impact process could indicate a different problem: automation bias or ineffective human challenge.
Governance should therefore pay attention not only to what AI systems do, but also to how people behave around them.
3. Control interventions
Runtime and workflow controls create another valuable source of information.
Suppose a guardrail successfully blocks an AI-enabled process from calling a restricted tool. The immediate conclusion may be that the control worked.
That is correct, but incomplete.
If the same control intervened twice last quarter and 200 times this week, the important signal is no longer simply control effectiveness. Governance should ask why the system is increasingly reaching the boundary in the first place.
A control that repeatedly prevents failure may also be providing evidence of increasing underlying risk.
4. Near misses
Near misses sit even closer to the eventual incident.
An incorrect AI-generated action may be caught by a human before execution. A transaction may be stopped just before completion. An inappropriate output may reach an internal user without reaching a customer.
Traditional reporting can treat these as examples of controls working successfully. A stronger approach also treats them as data.
The NIST AI Risk Management Framework Playbook explicitly recommends monitoring for unexpected behaviour and near misses, and maintaining information on errors, near misses, incidents and negative impacts. Near misses can reveal plausible failure pathways without requiring organisations to experience the full consequence first.
5. Material incidents
At the top of the stack are events organisations already know how to recognise: customer harm, financial impact, regulatory breaches, security events, operational disruption or other material failures.
These remain critical and should continue to drive formal incident response.
The OECD AI Incidents and Hazards Monitor reflects the growing global focus on learning from actual AI incidents and hazards. The opportunity inside an enterprise is to complement that incident learning with earlier internal signals.
The goal of stronger governance is to learn as much as possible from the lower layers before risk repeatedly reaches the top of the stack.
Figure 1. AI Governance Signal Stack. Original framework by Kunal Deore.
A green dashboard can still hide increasing risk
This creates an important challenge for executive reporting.
Many governance dashboards naturally emphasise process completion: the percentage of systems assessed, reviews completed on time, policies acknowledged, controls tested or incidents recorded.
Those measures tell leaders whether governance activity is occurring. They do not necessarily tell leaders whether risk is increasing.
Consider two AI systems that both report zero incidents.
System A has stable behaviour, few exceptions, consistent human challenge and minimal guardrail intervention.
System B also has zero incidents, but human overrides have doubled, control interventions have increased sharply and several near misses have occurred.
An incident-led dashboard could present both as green.
A risk-sensing view would not.
From individual metrics to patterns
The answer is not to build an enormous dashboard containing every observable AI metric.
More data can easily create more noise.
Instead, organisations should identify a small number of signals linked to meaningful risk hypotheses and examine them together.
Override rates, policy exceptions, control interventions, unusual system behaviour, escalation patterns and near misses become more useful when viewed as a connected story rather than as six independent KPIs.
Context matters.
A rise in overrides could indicate deteriorating AI quality, or it could reflect a deliberate policy change. Increased guardrail interventions might indicate emerging risk, or simply higher transaction volume.
Weak signals require interpretation, not automatic escalation.
That is why risk sensing should complement rather than replace accountable human judgement.
Governance should ask a different question
As AI governance matures, leadership conversations will need to move beyond: How many AI incidents did we have?
They should increasingly ask: Where is AI risk beginning to move before it becomes an incident?
Which controls are intervening more frequently? Where are people increasingly disagreeing with AI? Which exceptions are recurring? Are several systems exhibiting similar changes because they share a model, vendor, dataset or infrastructure dependency?
And are those signals changing quickly enough that existing controls, operating boundaries or oversight should be reconsidered?
These questions shift governance from retrospective reporting toward forward-looking decision support.
The next stage of AI governance is risk sensing
AI governance will always need policies, inventories, risk assessments, controls and incident management.
But as AI becomes more dynamic and embedded in enterprise operations, organisations also need the ability to detect deterioration between formal assessments and before visible failure.
The opportunity is not to predict every AI incident. That would be unrealistic.
It is to increase the probability that organisations recognise meaningful changes in risk while there is still time to investigate, intervene or reduce exposure.
The most mature AI governance functions may therefore not be the ones producing the largest number of dashboards.
They will be the ones capable of connecting operational evidence into signals leaders can act on.
Because effective governance is not demonstrated simply when nothing has gone wrong.
It is demonstrated when the organisation can see risk changing before something does.
