When we think about the risks associated with artificial intelligence, the focus typically falls on the technology and models itself: data quality, unfair bias, explainability, robustness, fairness, cyber security, privacy and lifecycle controls. Of course, these remain essential. However, as AI systems become more fluent, persuasive and embedded into everyday work through the use of agents and agentic systems, a different class of risk is emerging, not from the models themselves, but from how people interact with them.
The increasing risk is no longer that an AI system produces a confidently wrong output, but that people increasingly treat these systems as authoritative, reliable and “colleague‑like”, rather than as probabilistic machines that can fail. This shift in human behaviour fundamentally changes the enterprise risk profile of AI.
Pressure for efficiency
There is a growing tension between efficiency pressures and accountability. Yes, AI can significantly accelerate work, but validating AI outputs takes time, often eroding the very efficiency gains businesses are seeking. Under pressure to deliver more, employees may rationally choose speed over scrutiny. Trusting the technology delivers efficiency; validating the outputs delivers assurance. Organisations are often making their employees choose between the two.
Complacency
As these tools become more confident, fluent and context‑aware, people can start to over‑rely on them, especially when under time pressure. If AI outputs have been accurate repeatedly in the past, users may assume they will stay right and skip validation. Complacency also shows up as “purpose creep”: using models beyond their intended task or knowledge base, increasing the risk of inaccuracy.
Shadow AI
A related and increasingly common issue is the use of shadow AI. When employees cannot access useful, approved tools through official channels, often because rollout is slow, access is limited, or the tools do not meet their actual needs, they may look for workarounds including an increasing use of public models, personal accounts or unapproved tools to get the job done.
Controls
When misuse occurs, we often believe existing controls and frameworks will be enough. However, they typically assume intentional compliance. In reality, behaviour is shaped by time pressure, incentives, social signalling and cognitive shortcuts—so policies, training and governance alone are not sufficient.
How to manage the behavioural risks
We need to ensure not only that AI models can be trusted, but that the people using them understand and manage the associated risks.
Training may feel like the obvious answer when it comes to using AI effectively. But many of the growing risks are behavioural, and they need to be tackled by understanding how people use AI day to day — not just what they’re told about it.
Training will always play a role, but on its own, it is not enough. It assumes people will consistently apply what they have learned, even when they are busy, under pressure or being pushed to deliver faster. There are several control techniques that firms can adopt to reduce behavioural risks:
- To reduce complacency, organisations need to make sure AI use is not positioned purely as a way firms can move faster, but as something that still requires judgement. Giving people the space and permission to pause, sense check and challenge outputs makes a real difference.
- Incentives also matter. If performance measures quietly reward speed over quality, people will naturally defer to AI outputs and move on. Rebalancing expectations so that checking, validating and applying judgement are seen as part of the job helps counter automation bias without relying on perfect individual behaviour.
- Setting clear boundaries around AI use also helps reduce risk and the use of shadow AI. Many issues arise when models are used beyond their original purpose, not because people are deliberately ignoring rules, but because those rules are unclear or disconnected from daily work. Making intended use cases explicit, and embedding those boundaries directly into workflows, reduces the need for people to make judgement calls in the moment about what is or is not acceptable. Making safe, effective AI genuinely accessible, also helps mitigate risk as employees are not pushed outside of their organisation’s controls in the first place.
- Focusing on system design can be beneficial. Many AI tools are deliberately frictionless to encourage adoption, but removing all friction can increase blind trust. Introducing small prompts that encourage reflection such as asking users to review, justify or sanity‑check outputs, can help keep people stay engaged. Designing AI systems with full auditability also matters. When decisions, prompts and outputs are transparently logged and reviewable, it reinforces accountability and makes reflective nudges more meaningful.
- Critical thinking culture. Because AI behaves very differently from traditional, static technology, people need support in how to work with it day to day. This is less about deep technical training and more about building confidence to question outputs, recognise uncertainty and know when escalation or human judgement is needed. These expectations should vary by role, with stronger safeguards where decisions carry greater impact.
- Leadership: leadership behaviour matters more than is often recognised. When leaders consistently ask: “does this make sense?” or “what judgement was applied?” rather than defaulting to “what did the AI say?”, they set a tone that helps to discourage complacency.
As businesses scale AI across the enterprise, the most significant risks may be the least visible: over‑trust, complacency, purpose creep, use of shadow AI and performative use driven by incentives rather than outcomes. Effective management of AI risks therefore requires more than trustworthy models and associated data; it requires a deeper understanding of how people behave when powerful, persuasive tools are placed in their hands. Only by addressing both sides of the equation — technology and human behaviour — can organisations realise the benefits of AI without introducing new and unmanaged risks.
