Most AI-agent demos share one convenient property: nothing bad happens when they’re wrong. A chatbot misreads a question, a summariser drops a line, and the worst outcome is a mildly annoyed user. That calculus inverts the moment an agent does load-bearing work inside a regulated business – work a compliance officer signs off on and a regulator may one day audit.
Over the past two years, our team at Redberry built exactly that. Inside PP Mobius – the protection-advice platform we build and run for Tavistock Protect, part of the FCA-regulated Tavistock Investments Group – AI agents now do real work inside a formal compliance review. When a financial adviser submits a client fact find, agents analyse it as part of the review that decides whether the advice a real person received about income protection or life cover was suitable. This is our field report on what that actually took: the architecture, the design decisions regulation forced on us, and why “AI in regulated industries” turns out to be less about the model and almost entirely about what you build around it.
Why doesn’t compliance review scale on people alone?
Compliance review is the bottleneck because every piece of regulated advice needs one, and it is judgment-dense, repetitive, and human-rate-limited. In protection advice, every fact find – the structured record of a client’s circumstances, needs, and the recommendation made – passes a review that checks dozens of items across six areas: is the client data complete and consistent, does the income-protection recommendation fit the client’s situation, were life and critical-illness needs assessed, was the required courtesy call made, are the ancillary regulatory items in order.
Some of those checks are mechanical; some demand real judgment; all take time. At the scale Tavistock Protect operates – its compliance function processes roughly 600 fact finds a month – throughput becomes a hard ceiling on the business. You cannot scale regulated advice to more clients without scaling compliance, and you cannot scale compliance by hiring in a straight line without scaling cost and risk with it.
The obvious move is to point one large language model at the fact find and ask “is this compliant?” That is precisely the wrong design. A single monolithic verdict is unauditable, unexplainable, and impossible to trust selectively: if the model is right about nineteen things and wrong about the twentieth, you have no way to tell which. So we built the opposite.
The architecture: one micro-agent per check, not one big brain
Our core design rule is simple to state and hard to hold: one check, one agent, one traceable result. When a fact find is submitted in PP Mobius, the system initialises a structured compliance checklist, and every item on it is one of three kinds.
System checks are deterministic business rules. If a rule can be expressed in code – a duplicate-client test, an AML-completion gate, a disclosure-document check – it stays in code. No model, no ambiguity, no confidence score to interpret. This sounds obvious and is a genuine discipline: the pressure to “just let the AI handle it” is constant, and resisting it keeps the AI surface area small and reviewable.
AI checks are where the agents come in. Each is handled by its own micro-agent that reads the fact find independently and returns a result with a confidence score and a full trace of its reasoning. Independence is the point: the agent assessing fact-find completeness shares no reasoning chain with the agent checking the income-protection recommendation, so an error in one cannot cascade into the other.
Manual checks stay human. Some assessments require a compliance reviewer to weigh judgment and record a decision. Pretending otherwise would be both bad engineering and bad regulation.
Technically, the engine is built on the Laravel AI SDK. Each check type is a discrete class behind a shared interface, so adding a check means implementing one class, not re-architecting the pipeline. When a review starts, the system dispatches an individual micro-agent per check, each running in its own queue job with its own execution trace and its own confidence assessment; a final aggregation agent synthesises the individual results into an overall summary. Because every agent runs asynchronously on the queue, none of it blocks the interface – results stream to the reviewer’s screen as they complete. Compliance reviewers watch the checklist fill in live, in real time, with no page refresh. The check groups mirror a real regulated protection-advice review – preliminary client-data validation, fact-find completeness, income-protection checks, life and critical-illness checks, courtesy-call requirements, and ancillary regulatory items – and joint applications get independently tracked compliance tabs per client, because two applicants mean two sets of obligations.
What regulation forced us to build – and why it made the system better
In a regulated industry, the agent’s answer is the least interesting part of its output. What an FCA-regulated compliance process effectively demands is that you can reconstruct how any assessment was reached, potentially years later. That single requirement shaped three decisions that, in hindsight, we would now make on any serious agent system.
Every check result carries its own evidence. Evidence items, reasoning steps, trace outputs, and a confidence level are stored against each individual result. A compliance officer – or an auditor – can open any AI check and see exactly what the agent looked at and how it reasoned. Confidence scoring isn’t decoration: a low-confidence result is a signal to route that item to a human, not a number nobody reads. The agent doesn’t hand you a verdict; it hands you a worked file.
Reviews are versioned and immutable. Compliance can create a new version of a review – selectively re-running the automated checks, resetting the manual ones – without losing anything. Every prior version is preserved as a read-only snapshot with its own isolated activity log. Nothing is overwritten, ever. When your users answer to a regulator, “we updated the record” is not a sentence you get to say; “we created version three, and versions one and two are frozen” is.
Everything exports for audit defensibility. Any version of any review exports as a structured PDF built specifically for FCA audit defensibility – version number, check statuses, reviewer notes, and confidence levels included. The artefact a regulator sees is the same record the reviewer worked from. The discovery that surprised us: these constraints didn’t fight the agent architecture, they validated it. Micro-agents with isolated traces are auditable in a way a single monolithic model call can never be. Regulation, in effect, pushed us toward better engineering.
The honest caveats
We would be misrepresenting the work if we claimed the AI checks replaced the compliance team – they didn’t, and they shouldn’t. Redberry is not a regulated entity; Tavistock operates under the FCA; the agents assist and a human always makes the regulated decision. The system’s job is to take on the checks that are reliably automatable, surface its reasoning transparently, and leave the genuine judgment calls to people – faster, and with better evidence in front of them. We introduced checks manually first and automated each only once it earned trust, shipping roughly a new one every two weeks; today a growing share of the review is agent-assisted, on a path toward automating around fifty checks and 200–250 data points. Non-determinism is a real property of LLM systems, and the mitigations – deterministic rules where possible, confidence scoring, human-in-the-loop review, immutable audit trails – are not optional extras bolted on at the end. They are the product.
The takeaway
The lesson from PP Mobius isn’t that AI agents can work in regulated industries – it’s how they have to. Decompose the problem into small, independent, individually traceable agents. Keep deterministic logic deterministic. Store the reasoning, not just the verdict. Version everything immutably. Build the audit artefact as a first-class feature, not an afterthought.
If you’re exploring how agentic systems could take on operational workflows in your own product – regulated or otherwise – this decomposed, evidence-first architecture is where we’d start. It’s the approach behind our AI agent development work at Redberry, where we help teams validate agent automation on real workflows before scaling it. The models will keep getting better. The scaffolding around them – traceability, versioning, human oversight – is what turns them from demos into infrastructure.
