Cyber Security

AI Now Writes the Code. Security Testing Still Assumes a Human Did.

There is a mismatch sitting inside most engineering organisations right now, and it is not the one that gets discussed at conferences.

The writing side of software development has been rebuilt in about eighteen months. Agents scaffold services, generate endpoints, wire up authentication, produce migrations, and open pull requests. Teams that measured output in features per quarter now measure it in merges per day. Whatever anyone thinks about the quality of that output, the throughput change is real and it is not reversing.

The verification side has not moved at all. Most companies still assess the security of their applications the way they did in 2019: an annual penetration test, a scanner running on a schedule, and a code review process that assumes a human wrote the diff and another human will read it carefully. Those assumptions were reasonable when a team shipped every two weeks. They are decorative when the same team ships eleven times before lunch.

This is not an argument that AI writes insecure code. It is an argument about arithmetic.

Why AI-generated code fails differently

The instinct is to worry that models produce vulnerable patterns, and sometimes they do. But that is the tractable half of the problem, because vulnerable patterns are exactly what static analysis and code review are good at catching. A hardcoded credential, an unparameterised query, a disabled certificate check: these have recognisable shapes, they show up in a diff, and both a linter and a reviewer will flag them.

The harder failure is structural, and it comes from consistency rather than carelessness.

When an agent generates forty CRUD endpoints from a schema, all forty are well-typed, documented, and stylistically identical. Authentication is handled centrally by middleware, so every one of them correctly requires a valid token. Authorisation is a different thing entirely: it is per-object and per-tenant, it depends on what your product means by ownership, and it has to be written into each handler. Thirty-nine handlers get it. One does not, because the model had no way to know that the `organisation_id` on that particular record is the boundary that matters.

That single endpoint is now a data breach with an ID parameter in front of it, and it looks exactly like its thirty-nine safe neighbours. There is nothing anomalous in the diff. This is the practical limit of AI code review as a control: a reviewer, human or machine, is reading a change in isolation and asking whether it is well-formed. The question that matters is whether the deployed system as a whole still enforces a boundary, and that question cannot be answered by looking at a diff.

Volume makes it worse in a way that is easy to underestimate. If a reviewer catches 95 percent of authorisation mistakes, that is excellent work at ten changes a week and a steady leak at four hundred.

The category that survives every automated check

There is reasonably hard evidence about which flaws actually make it to production, and it points at the same place.

Across 47,291 exploitation-validated findings from 3,847 web applications and APIs tested between October 2025 and March 2026, broken access control was the single most common finding class, present in 42 percent of all applications assessed. More usefully for this argument: scans run with an active authenticated session found 3.4 times more vulnerabilities than scans of the same applications run without one. That dataset is published by Penetrify under a CC BY 4.0 licence.

The 3.4x figure is the interesting number. It says that a conventional unauthenticated scan is not giving you a partial view of your attack surface. It is giving you a small minority of it, and the part it systematically omits is the part that requires reasoning about identity. Nearly every instance of one user reaching another user’s data was invisible until testing logged in with real credentials and then tried, deliberately, to cross a tenant boundary.

This is the mechanical reason these bugs survive. It is not a gap in anyone’s signature database. It is a missing test oracle. A scanner sees an HTTP 200 with well-formed JSON and records a successful request. It has no way to know the invoice in that response belongs to a different company. Establishing that requires two identities, a model of which objects belong to whom, and an attempt to violate that model on purpose.

Agentic development happens to concentrate risk precisely there, in per-object authorisation across a large and uniform surface, which is the one category that both AI code review and traditional scanning are structurally unable to cover.

What changes when the testing side becomes agentic too

The reason this gap stayed open for so long is economic rather than technical. Everyone in security has known for twenty years that authorisation flaws need someone to log in as two users and try things. That is skilled manual work, it costs $15,000 to $50,000 per engagement, it takes a week or three to schedule and deliver, and so it happens once or twice a year. The result is a very deep look at one afternoon in one quarter, which then ages badly against a codebase being rewritten continuously.

The same capability that changed the writing side is now available on the testing side, and it changes that trade-off rather than eliminating the need for human testers. An agent can hold credentials for multiple accounts, build a model of what objects exist and who owns them, attempt to cross those boundaries, and report only what it managed to actually demonstrate. That last constraint is the important one, because it is the difference between a finding and a guess. AI penetration testing that validates by exploitation produces a list of things that were proven, with the requests and the retrieved data attached, rather than a list of conditions that might be concerning.

What this does not do is replace skilled humans. Novel business logic abuse, chained attacks that depend on understanding a company’s commercial model, and anything requiring genuine creativity remain human work, and any vendor claiming otherwise should be treated accordingly. What it does is move depth from an annual budget line to something that can run at the cadence you deploy at.

For an organisation whose code is now substantially machine-generated, that is the relevant question. Not whether the agent writing the code is trustworthy, but whether verification runs often enough to be meaningful against the rate of change.

The practical version

Three things are worth doing, in order of how much they return relative to the effort.

Test with real sessions and more than one account. If your security testing runs unauthenticated, it is examining your login page and your marketing site. Ask your vendor directly whether testing maintains an authenticated session and whether it uses at least two distinct accounts, because one set of credentials cannot detect the most common serious flaw category. This question alone eliminates a surprising share of products.

Match verification cadence to deployment cadence. An annual assessment against continuous deployment leaves fifty-one weeks of unreviewed change in production. Wiring security testing into the deployment pipeline is now a realistic engineering task rather than an enterprise programme, and it is the only arrangement where the words “we test our applications” describe something true about what is currently running.

Require proof, not inference. Ask to see a sample finding from whatever tool or vendor you use. If the evidence section contains a request, a response, and the data that was actually retrieved, you are looking at something demonstrated. If it contains a description of what an attacker might theoretically achieve, you are looking at a hypothesis that someone on your team now has to spend an afternoon confirming or dismissing.

None of that is novel security thinking. It is standard practice that was previously too expensive to apply continuously.

The uncomfortable part is what happens if the mismatch stays open. Organisations are increasing code output by an order of magnitude while holding verification flat, and the categories of flaw that agentic development concentrates are exactly the categories that neither AI code review nor conventional scanning can see. That produces a growing volume of undetected authorisation bugs in production, and there is no reason to expect the market to discover this gently.

Viktor Bulanek is the founder of Penetrify, an autonomous penetration testing platform for development teams, based in Brno, Czech Republic. The vulnerability statistics cited here come from an aggregated dataset of 47,291 exploitation-validated findings, published under CC BY 4.0 at penetrify.cloud/en/stats.

Author

  • I am Erika Balla, a technology journalist and content specialist with over 5 years of experience covering advancements in AI, software development, and digital innovation. With a foundation in graphic design and a strong focus on research-driven writing, I create accurate, accessible, and engaging articles that break down complex technical concepts and highlight their real-world impact.

    View all posts

Related Articles

Back to top button