Across workplaces, employees are increasingly turning to AI for quick, everyday tasks. But as adoption accelerates, so too does the number of attack vectors for financial criminals, with compliance not growing at the same rate as uptake.
Deepfake authorisation, automated prompt injection and synthetic identity generation are just some of the vulnerabilities emerging as businesses introduce AI faster than they can prepare their people to manage it.
This growing unease is reflected in a recent 250% surge in demand for responsible AI training content. There’s clear evidence of corporate anxiety about the software, and many businesses are grappling with the implications of rapid AI adoption, from financial and operational risk to workforce disruption. Organisations are often clued up on the advantages of using AI, but the true challenge comes with properly understanding the risks that come with it.
This is particularly important when AI is introduced into compliance – often as a cost-cutting measure for staff training. Used well it can make training more efficient, personalising learning paths, supporting interactive roleplay simulations and handling routine tasks. But efficiency cannot come at the expense of judgement.
AI can support and is useful for simple and low-risk processes such as data sorting, personalisation and routine learning, but human judgement remains essential for more complex decisions such as responses to financial crime and ethics.
The answer is not to remove AI from compliance altogether, but to establish clear human-in-the-loop (HITL) governance, where technology supports decision-making without becoming the final decision-maker.
The common sense deficit
AI is a master of pattern recognition, but a novice when it comes to common sense. Its strength lies in identifying relationships in the data it has been trained on. Its weakness is knowing what to do when reality produces something it has never come across before.
This creates a particular vulnerability in financial crime. Fraudsters deliberately look for gaps in established controls, exploiting situations that don’t resemble historical patterns. Sophisticated financial crime relies on context, intent and nuance – precisely the areas where pattern matching alone falls short.
This is where human judgement becomes a critical line of defence. A person can recognise that something doesn’t quite add up, even when the available data doesn’t contain any obvious precedent. Historical algorithms are no replacement for human intuition and contextual reasoning which pick up behavioural anomalies. An algorithm is likely to struggle when faced with scenarios outside of the parameters of its training.
The consequences of over-trusting automated systems can already be seen elsewhere. A major UK supermarket brand recently paused its live AI-assisted facial recognition system after a shopper was wrongly identified as a shoplifter, accused and ejected from the store. The incident followed a similar case at another branch and prompted a company-wide review of the technology.
While this was not a financial crime incident, it illustrates a broader problem with automated decision-making: when people assume that an algorithm must be right, an incorrect output can quickly become an incorrect human decision. The technology didn’t need to make a perfect prediction to create harm; it simply needed people to trust its prediction without sufficient challenge.
The same principle applies to compliance. AI systems can identify patterns at extraordinary speed, but they cannot be allowed to become a substitute for professional curiosity. Clear oversight, continuous testing and the confidence to question an output are essential to preventing vulnerabilities from becoming exploitable weaknesses.
Ethics, ESG constraints and algorithmic bias
Another major risk of unchecked AI is the creation and acceptance of patterns that are potentially unethical. Failing to continuously test and improve training data can cause AI to replicate discriminatory patterns and introduce algorithmic bias.
Using a system that has unchecked algorithmic bias for onboarding, decision-making or risk profiling increases the potential for regulatory, legal and reputational risk to fester. The ethical and legal liabilities of this cannot be overstated. Algorithmic bias could, for example, lead to the unfair flagging of specific demographics or geographic regions, embedding discriminatory outcomes into processes that are increasingly difficult to scrutinise at scale.
The growing scrutiny over the environmental impact and energy usage of AI is another factor. We know that Large Language Models (LLMs) complicate corporate ESG targets because of their hidden resource footprints. There is clear tension between AI’s energy and compute demands and corporate sustainability commitments.
For this reason, AI should remain a supportive tool rather than automatically becoming the default for every process within a business. Responsible adoption means understanding not only what AI can deliver, but where its wider ethical and environmental costs may outweigh its benefits.
The regulatory reality and why humans hold the reins
Regulation makes one thing all too clear. Accountability cannot be outsourced to an algorithm.
The FCA and the Bank of England hold human professionals accountable for effective compliance oversight. The ‘algorithm made a mistake’ will  not be viewed as an appropriate defence. Organisations of all sizes need to be training their people to exercise appropriate judgement, challenge systems and take responsibility for the decisions being made. One such policy, the Senior Managers and Certification Regime (SMCR), places direct personal accountability on senior leaders for technological and compliance failures.
The repercussions of failing to do so can be significant. Companies that have failed to follow procedures or make a major error are liable for large financial penalties. In December 2025, the FCA fined a major national bank £44 million for inadequate anti-financial crime systems and controls.
Although the organisation had previously warned about its conduct, it failed to make improvements in time. The case demonstrates why technology cannot be treated as a substitute for effective oversight. Trained staff remain the necessary circuit breaker capable of identifying weaknesses, intervening when controls fail and preventing issues from escalating into substantial fines.
Ultimately, failures in systems and controls come back to the people responsible for overseeing them rather than the technology itself.
How to transform people from passive observers into fail-safes
Without extensive training on the proper usage of AI, staff are likely to lapse into automation complacency. This can turn people into passive observers who blindly trust machine output and become more susceptible to bias.
Imagine an analyst approving a flagged transaction simply because an AI system has assigned it a low-risk score. The problem isn’t necessarily that the technology has made a decision, it’s that nobody has challenged the decision.
Training therefore needs to empower people to challenge, interrogate and override AI where necessary.
Building human resilience among compliance leaders requires scenario-based training, continuous prompt-auditing and regular red-teaming. Employees need to understand both the capabilities and deficiencies of AI so that they can recognise when an output should be questioned rather than accepted.
This moves training beyond simply teaching people how to use AI. It equips compliance teams to actively interrogate AI outputs, identify exploited loopholes and recognise when technology has reached the limits of its capability. Most importantly, it gives them the tools to act as the company’s guardians.
True compliance resilience relies on highly trained and vigilant professionals who can exercise human judgement alongside machine speed.
What does the future of human agency in the AI era look like?
AI is an exceptional engine for efficiency, but operating it without effective governance leaves critical security loopholes.
As adoption accelerates, leaders will need to actively monitor machine outputs and anticipate novel attack vectors rather than waiting for technology to fail before intervening.
Ultimately, it is not algorithms that carry the accountability – its people. AI can provide speed and efficiency but all organisations need to invest in the humans at the core of widespread and sustainable AI adoption.
Prioritising upskilling will enable businesses to build true operational resilience, give their workforce the knowledge and confidence to challenge AI, step-in where necessary and act as a fail-safe against future, unknown risks.
Machine learning may move at speed, but human judgment remains the last line of defence.
