Businesses are beginning to hand AI authority that once belonged to people, from approving customers and moving data to executing decisions. Done well, this could transform AI from a productivity assistant into a genuine business and operational enabler. But greater autonomy requires greater confidence and rigorous testing, governance and monitoring can provide the evidence businesses need to safely expand what AI is permitted to do. At the same time insurance can help transfer some of the residual risk that remains. Together, these can enable businesses to use agentic AI more extensively and with greater confidence.
The UK Government’s rejection of proposals for a central AI “kill switch” reinforces why businesses need to manage this exposure themselves, particularly when models and infrastructure operate across borders and legislation will inevitably struggle to keep pace with the speed at which the technology is developing.
Autonomy changes the scale of failure as an agent managing a customer journey might create documents, file information and send responses, so any departure from its instructions could affect hundreds of cases before the problem is discovered. The exposure becomes more serious when AI makes know-your-customer, compliance, credit or legally significant decisions. These processes traditionally involve different people, control points and approvals. Combining them in one autonomous workflow can remove that separation of duties and allow a single weakness to travel from the beginning of a process to the end.
Businesses should therefore assess AI exposure through three questions – what authority does the system have, how far can its actions reach and can those actions be reversed? A tool answering routine enquiries presents a very different risk from one able to approve customers, move data or execute transactions.
Insurance should begin before deployment
Too often we see insurance is considered only after an AI model has been selected, integrated and given access to live systems, when insurability should be examined while the use case and its controls are still being designed.
From an insurance perspective we need to understand what the model will do, the decisions it can make, the data it can access and the maximum credible impact of failure. If a business cannot answer those questions, it may not understand the risk well enough to operate it, let alone transfer it.
Authority limits, audit trails, testing and escalation procedures are the foundations of responsible AI governance. Insurance should protect against residual risk after safeguards are applied as it is not permission to deploy an uncontrolled system.
Businesses should not assume every AI loss will fall neatly within cyber or professional indemnity insurance. Cyber policies have traditionally focused on malicious attacks, security breaches and other defined events, which means restoration costs may not necessarily satisfy the policy trigger if an autonomous model malfunctions and deletes, moves or alters data without malicious intervention.
Professional indemnity insurance may respond when a customer suffers loss because of a failure in professional services, although this will depend on the policy wording and liability involved. The difficulty is that an AI incident could resemble a cyber loss, technology failure, operational error and professional liability claim without fitting comfortably within any one category.
Responsibility may be shared by the developer, provider, integrator and deploying business. Contracts may allocate liability, but cannot guarantee it is insured. Extensive autonomy, weak controls and unauditable actions also make exposure harder to price.
Managing AI risk effectively will require technical testing, continuous monitoring and insurance to work together. Red teaming can establish whether a model can be persuaded to disclose information, override instructions or exceed its intended purpose, while ongoing monitoring can reveal behavioural changes and insurance can respond to the defined risks that remain.
Our work with AI security specialist Vulcan is moving in precisely this direction by connecting technical assessment with a clearer evaluation of insurability. Insurance is not, by itself, the answer to AI risk, the aim is to help organisations understand and control that risk before transferring the part they cannot eliminate.
Although no organisation can remove every risk associated with autonomous AI, businesses can determine how much authority these systems receive, how rigorously they are tested and how quickly people can intervene when something goes wrong. Those decisions will separate organisations adopting agentic AI responsibly from those simply automating risks their controls cannot contain and their insurance was never designed to cover.
