AI-generated code is a powerful software development tool and, in some respects, the natural successor to low-code or no-code. Professional software developers use it to test ideas, accelerate delivery, and roll out new features. Alongside them, citizen developers increasingly use agentic coding tools to build integrations, reports, and multi-step finance automations for their organisations.
The result is a much shorter path from original concept to a functioning financial workflow—plus improved automation for some of the finance sector’s most time-consuming, manual tasks.
By making software development more accessible, AI-generated code can be genuinely democratising—and that is, in essence, a good thing. The flip side is that people can create and deploy software without fully understanding the code or knowing how to rigorously test and query it.
Experienced developers are less likely to lack those skills, but they face a different risk. The promise of shorter development cycles may create pressure to accept AI-generated code without the checks and balances they would apply to their own—or a peer’s—work.
A perfect storm
Defects can arise in any software, but AI-generated code creates a particular risk when the person deploying it cannot fully explain, check, or revise what it produces. That risk increases when the finance team reviewing the output has little or no experience of the manual steps behind the automated process—making a plausible but incorrect result harder to recognise.
It’s a perfect storm. Finance and accounting processes operate within a very small margin of error. Miscalculating an organisation’s year-end position by even a few per cent could mean reporting a profit when it made a loss, calculating the wrong tax liability, or giving decision-makers a distorted view of the business.
As such, whether AI-generated code is deployed by an established software provider or created inside an organisation, the resulting solution must still withstand the same scrutiny applied to human-written code. That means establishing how it behaves before it enters full use, knowing where it might fail, and determining whether it can be trusted.
Yet even with those checks in place, a technically successful workflow can still produce a financially incorrect result. For that reason, those responsible for the workflow must also understand which data fields it uses, where the information comes from, whether it draws on posted or draft transactions, which period it covers, and what calculations or transformations it applies.
Working code is not necessarily safe code
Financial correctness is only one test. Code may function as intended and produce an accurate result—but it can still introduce a security risk. In one controlled study, participants using an AI coding assistant produced less secure code than those working without one, while also being more likely to believe that their code was secure. The findings do not mean all AI-generated code is unsafe, but they show why speed, functionality, and confidence are not substitutes for security review. (Perry et al., Do Users Write More Insecure Code with AI Assistants?)
For established accounting and finance software providers, AI-generated code may become part of a product that handles supplier information, invoices, payment data, approvals, tax records, reporting, and audit trails across multiple customer organisations.
A weakness in authentication, data validation, or an integration could therefore expose confidential information, provide an unintended route into other systems, or allow an error to spread at scale. For that reason, AI-assisted development must sit inside strong architecture and governance—including peer review, security testing, and controlled release processes.
Citizen developers inside organisations face a similar risk, but often without the same development infrastructure. An automation might be given access to accounting records, shared files, email accounts, reporting data, or credentials so that it can complete a multi-step task with little or no human intervention once initiated—and potentially without its creator having substantial coding expertise.
Its creator may not think of this as deploying software. Yet broad permissions, embedded credentials, or an inadequately tested integration can create an unexpected route into systems that were otherwise secure. A workflow with insufficient access may calculate a result from an incomplete set of records. But one with excessive access may expose restricted information or initiate an action the user is not authorised to take.
Guardrails must exist at every layer
AI is not the enemy, but guardrails cannot be a single final check performed after the code has been written. They must operate throughout development, inside the finished workflow, and at the point where users decide whether they can trust the output. No system should serve as the sole check on its own work, particularly where AI-generated code or AI-powered automation is involved.
For established providers, guardrails must continue through review, release, and ongoing operation. Clear documentation and ownership allow development teams to understand and maintain AI-assisted code, while monitoring and staged rollout make unexpected behaviour easier to detect and reverse.
Organisations that champion citizen developers need a proportionate version of the same discipline. Automations should be tested away from live financial data, given only the access required for their specific task, and prevented from taking consequential actions without approval. This governance cannot be assumed. The UK Business Data Survey 2026 found that only 17% of businesses using AI had a policy or guidance covering its use or development. Furthermore, formal written policies were far more common among large businesses than smaller organisations.
Supplier trust also depends on making uncertainty visible rather than offering token reassurance. Output confidence scores can help, but only when users understand what they mean and are encouraged to question results rather than accepting them automatically.
In automated document capture workflows, for example, confidence thresholds can route uncertain entries for human verification before they enter the ledger. Organisations can set thresholds that reflect the financial or operational risk involved, informed by performance established through testing and ongoing monitoring. But even then, a confidence score is not proof of accuracy. Its value lies in directing human attention to where review is needed.
Where AI interprets data or generates an output, the workflow should allow it to return an explicit unknown or error rather than forcing a plausible answer. Validation loops can compare its output with source data or expected totals, while exceptions can be sent to a human reviewer. Approval stages should prevent an uncertain result from automatically changing a record, triggering a payment, or entering a board report.
Every workflow also needs a failure path, including traceable logs, a way to pause or reverse the process, and an alternative route when automation cannot complete safely.
Conclusion
AI-generated code is not inherently less trustworthy than human-written code. Trust is lost when no one can explain what the code does, test its financial logic, control the data it can access, or intervene when something goes wrong.
In the accounting and finance sector, trust also depends on preserving a clear source of truth. Automated outputs should not be treated as correct because they appear plausible—and users should be able to trace them back to the underlying accounting records. The foundations remain the same, whether the code sits inside an established product or an internally created workflow: review, validate, set out appropriate permissions, provide traceability, and encourage human accountability.
AI-generated code can accelerate delivery for developers and enable automations that remove repetitive work for users. That makes the technology worth embracing, provided its use is governed responsibly and the resulting code is subject to the same guardrails as human-written code. Ultimately, however, finance teams can only begin to trust AI-enabled software when the people building and deploying it can demonstrate that they understand the code, data, and accounting logic behind the automations.
