AI is fast becoming the silent backbone of the UK’s defence, government and critical infrastructure — threaded through decision-making, intelligence, and operations. But while the race has focused on squeezing out advantage, far less effort has gone into hardening these systems against attack and manipulation from malign state actors. That imbalance has resulted in a growing vulnerability that needs to be addressed now, before it is exploited.
As the UK and its allies continue to adopt AI‑augmented tools, the attack surface for adversaries is expanding in parallel. Many of these systems are deployed with the expectation that they will enhance efficiency, accuracy or situational awareness, but their increasing influence on decisions also amplifies the consequences if they are compromised. The risks are no longer limited to system outages or technical failures, and now include more subtle acts of manipulation, data poisoning and erosion of trust in the outputs themselves. These threats operate quietly, often without clear indicators, and exploit aspects of AI systems that traditional cyber‑security measures were not designed to address.
Without a stronger focus on resilience against adversaries, AI systems intended to strengthen national security may instead introduce new and poorly understood vulnerabilities. Addressing that gap requires that we recognise AI manipulation not as a hypothetical or niche technical problem, but as an emerging attack vector with a low barrier to entry and an immense potential for harm. How might these vulnerabilities arise? What would exploitation look like in practice? The UK and its allies must consider these questions seriously if they are to mitigate the risks before they manifest.
The attack surface is expanding
AI is quickly becoming embedded across UK organisations for everything from document triaging and administrative processes to intelligence analytics and logistics, though many argue that governance structures have not kept pace with adoption. Research from the Department for Science, Innovation and Technology (DSIT) found that AI use is growing, but adoption is uneven and there is a clear gap in skills, education and expertise.
Every new AI deployment tends to deepen institutional reliance on AI‑driven decisions. An article in the Financial Times pointed out that in many cases, systems have been added as “bolt-on” capabilities rather than integrated into a robust, end‑to‑end assurance framework. This creates an area of vulnerability: if an AI system is manipulated, degraded or subtly misaligned, the decision-maker using it may have no obvious warning signal. As more decisions are mediated, filtered or shaped by AI, the cost of a compromised output grows. That could mean anything from disrupted logistics to skewed threat assessments or widespread misinformation. The UK’s push for digital transformation is creating new operational capabilities, but it is also widening the target surface for adversaries who understand that influencing a model can often be more effective — and more accessible — than breaching a network.
Poisoning the well
Artificial intelligence systems are often described as highly complex, but some of their most damaging vulnerabilities stem from surprisingly simple techniques. Recent research from Anthropic has shown that just 250 deliberately crafted documents can reliably “poison” a model of any size during training, embedding hidden behaviours that activate only under specific conditions. This means an adversary does not need cutting‑edge capabilities or privileged access to the model itself, they simply need to influence the data a system will later learn from.
The implications for government and industry systems are profound. Many AI tools now used across the public sector like search engines and generative AI chatbots learn at least partly from publicly available data sources. If those sources are compromised before training, the manipulation becomes part of the model’s internal logic. There is no intrusion to detect, no log trail, and often no visible sign that anything is amiss. Importantly, many AI tools continuously learn and refresh their training data, so poisoning does not require access at the moment of initial model creation — an adversary can influence the system gradually by targeting the data it will absorb in later updates.
Because AI models frequently draw on broad public data ecosystems, an attacker can seed misinformation into open sources such as Wikipedia, forums, public datasets or archived news content. When the next training cycle pulls those sources into the model, the poisoned material is absorbed without any trace of manipulation. The range of ways an adversary can tamper with data before a model is trained is wide, poorly monitored, and fundamentally different from traditional cyber threats, as there is no need to breach a network when the model will willingly ingest the hostile material on its own.
A poisoned model could skew economic forecasting tools by altering parameter weights or suppressing warning signals. It could distort threat-assessment systems by downgrading particular indicators or misclassifying specific activities. It could even be tuned to amplify divisive content when used in public-facing applications like search engines and chatbots, in order to influence public perception or amplify societal fragmentation without any visible fingerprints.
Perhaps the most concerning aspect is how accessible these methods have become. Sophisticated cyber‑attacks have traditionally required specialist expertise, but data poisoning attacks can now be carried out with relatively limited resources and a modest understanding of data pipelines, making them more feasible at scale. This places attacks on training data increasingly within reach of non‑state actors, extremist groups, cyber‑criminal networks, and technically literate individuals.
How can institutions trust AI systems when their training data — much of it gathered automatically from the public internet — may already be incomplete, biased, or compromised? Without stronger data provenance controls, better training pipelines and systematic monitoring for manipulation, the UK risks deploying AI whose behaviour has been unintentionally or maliciously shaped long before it reaches an operational environment.
Erosion of trust
Deliberate manipulation of information environments is not new, but AI changes its scale, speed and strategic utility. For decades, states have used networks of fake or hijacked accounts to manipulate online discourse, in order to influence public opinion, shape political narratives and undermine democratic processes.
Public reporting has linked various state‑aligned groups to sophisticated influence operations, including bot networks, coordinated messaging campaigns and information‑laundering techniques, showing how sustained, organised manipulation can shape narratives at major scale. NBC recently covered a Graphika report that analysed multiple online influence operations, finding that many have used generative AI to flood the internet with “AI slop” as part of propaganda campaigns.
Historically, these efforts depended on vast volumes of human‑generated content to create the illusion of popularity or consensus. Generative AI removes this bottleneck, enabling sustained propaganda output at a scale that would previously have required thousands of operators.
Research from King’s College London suggests that AI‑enabled influence systems — capable of generating and distributing coordinated messaging via ‘AI propaganda factories’ and persistent automated personas — are now technically feasible and increasingly within operational reach for state actors.
These systems have the potential to rapidly produce convincing text, imagery and video, and then deploy them through bot networks that adapt in real time. This automation strengthens an adversary’s ability not only to spread false narratives, but to maintain the volume, timing and emotional framing needed to shape public discourse.
The strategic risk for the UK does not lie solely in individual pieces of misinformation, but in the cumulative erosion of confidence in government, defence and public institutions. If AI‑generated intelligence, summaries or threat assessments are suspected of being compromised even occasionally, their operational value is severely weakened. In this scenario, undermining trust could become a strategic objective in itself for adversaries that cannot match the UK’s technological capabilities directly.
We are still in the early days of AI adoption, and there are few publicly documented examples of data poisoning directly influencing government decisions. However, it is highly plausible that such scenarios could emerge in the near future.
Fending off the threat
Protecting the UK from AI‑driven manipulation requires treating vulnerability as a core national security issue rather than simply a technical or regulatory concern. The current frameworks governing AI, including the remit of the UK’s AI Safety Institute (AISI), already acknowledge risks around misuse, but direct manipulation in national security contexts is still not sufficiently defined or planned for. Strengthening this remit to explicitly cover training‑data poisoning, model manipulation and degradation would help bridge the gap between current safety policy and real‑world threats.
Industry also has a critical role to play. Organisations deploying AI in sensitive domains should assume that hostile manipulation is a legitimate risk. That means building adversarial resilience into systems from the outset by maintaining clear data‑provenance, offering essential training, continuously monitoring models for anomalous behaviour, and running red‑team exercises that simulate realistic manipulation scenarios. These measures are already standard practice in high‑assurance cyber environments such as classified networks and critical national infrastructure and extending them to AI is a necessary evolution.
Equally important is institutional awareness. Public and defence‑sector decision‑makers often rely on AI‑generated outputs under time pressure, and it is important to make sure that senior leaders, analysts and system users understand how those outputs might be deliberately degraded.
This threat differs fundamentally from the attacks the UK has spent decades preparing for. It does not require breaching a classified network or deploying elite‑grade cyber tools. It requires patience, open‑source‑style activity and an understanding of how AI systems learn, which hostile actors have already demonstrated repeatedly in other domains. The UK should assume it will face attempts to poison, manipulate or erode trust in its AI infrastructure, and start to establish the means to detect and withstand them.
