Adoption is no longer the most interesting number. The 2025 DORA State of AI-assisted Software Development report, drawing on survey responses from nearly 5,000 technology professionals, found that 90 percent use AI in their daily work and more than 80 percent believe it has made them more productive. Belief is doing a lot of work in that sentence. DORA also found that higher AI adoption correlates with improved delivery throughput and, at the same time, with worse delivery stability.
The independent research sharpens the point. METR’s 2025 randomized controlled trial put 16 experienced open-source maintainers on 246 real issues in their own mature repositories, and found they took 19 percent longer with AI tools while estimating they had been 20 percent faster. GitClear’s analysis of 211 million changed lines between 2020 and 2024 found code duplication rising from 8.3 percent to 12.3 percent and refactoring falling from around 25 percent of changes to under 10 percent. Stack Overflow’s 2025 survey found 84 percent of developers using or planning to use AI coding tools while 46 percent distrust their accuracy.
None of that argues against AI-assisted development. It argues that the pilot results most organizations use to justify scaling are measuring the wrong thing, in the wrong environment, at the wrong scale.
A pilot is not a small version of a program
Pilots are usually run on the friendliest surface available. A contained service, a small team, work with clear boundaries and few downstream consumers. Assistance performs well there, and the results are real.
Large engineering programs look nothing like that. Twenty parallel workstreams, dependencies that cross team boundaries, integrations with systems written before the current architecture existed, and a launch date that several other organizations have planned around. The constraint in that environment is almost never how fast someone can produce a first draft of a function. It is coordination, integration sequencing, and review capacity.
DORA’s finding that AI acts as an amplifier is the useful frame here. It magnifies what an organization already does. A program with weak integration testing and slow feedback loops gets more change volume flowing into the same fragile pipeline, which is exactly how throughput improves and stability degrades at the same time.
“The pilot tells you the tool works. It tells you almost nothing about whether your program can absorb what the tool produces.”
The prerequisites are organizational before they are technical
Leaders tend to treat this as a tooling rollout. Procure licenses, run enablement sessions, track usage. Usage then becomes the success metric, which is convenient and close to meaningless.
The prerequisites that actually gate scaled adoption are less exciting. Test coverage that can catch integration regressions rather than unit-level ones. Fast enough continuous integration that a developer finds out within minutes, not overnight. Clear ownership boundaries so that a change generated in one workstream does not silently alter behavior another team depends on. Version control discipline to be mature enough to make changes reversible.
DORA’s 2025 benchmarks suggest most organizations are not there. Only 9.4 percent of teams achieve lead times under an hour while 43.5 percent need more than a week. Only 8.5 percent hold change failure rates in the 0 to 2 percent range, while 39.5 percent sit above 16 percent. Accelerating code production into a pipeline with those characteristics increases the amount of work in flight without increasing the amount that reaches production safely.
“If your feedback loop takes a day, giving everyone a faster way to write code just means they wait a day to find out about more mistakes.”
Legacy integration is where the assistance gets thin
The METR result is easy to dismiss as a small study, and the sample was small. But the condition it tested matters for anyone running a legacy program: experienced engineers working in large, mature codebases they already know intimately.
That is precisely where assistance struggles. A model reading a legacy integration sees the code. It does not see the reason a particular field is populated in an unusual order, or the downstream consumer that breaks when a null becomes an empty string, or the migration that was abandoned halfway in 2019 and left two conventions in place. Suggestions come back plausible, idiomatic, and wrong in ways that pass review if review is moving quickly.
The practical response is to be explicit about where assistance is applied. Greenfield services, test scaffolding, migrations with mechanical patterns, and documentation reward it. Changes at legacy integration seams, in shared data contracts, or in anything with unclear ownership deserve slower treatment, regardless of what the tooling can generate.
“Assistance is strongest where the context is in the code. Legacy systems keep most of their context somewhere else, usually in people’s heads.”
Review capacity is the constraint nobody budgets for
DORA found that time saved during code creation frequently gets reallocated to auditing and verification. That work does not distribute evenly. It concentrates on senior engineers, who were already the bottleneck.
A program that doubles change volume without expanding review capacity has not become faster. It has moved the queue. Worse, reviewer fatigue tends to shift review from evaluating whether the approach is right to checking whether the code looks reasonable, which is the failure mode most likely to let a plausible but wrong change through.
Leaders should watch for the signals directly. Review latency trending up. Review depth trending down, visible in comments per changed line. Rework rate climbing. And the GitClear indicators, duplication and refactoring ratio, which reveal whether the codebase is being maintained or merely extended.
“Nobody puts review capacity in the productivity business case, and it is the first thing that breaks.”
Validate the gains, do not accept them
The measurement problem in the METR study is worth sitting with. Developers who had just been slowed down still reported being sped up. Self-reported productivity, satisfaction surveys, and license utilization are all measuring perception, and perception has been wrong by roughly 39 points in a controlled setting.
A durable validation approach looks at system outcomes rather than individual output. Did lead time from commit to production improve, or did more work simply pile up before the pipeline? Did failure rate change and rework rate stay flat while volume rose? Are defects found in integration testing and staging holding steady per unit of change, or growing? Is duplication rising and refactoring falling? And the question worth asking a year later: are engineers who joined during heavy assistance able to debug the systems they shipped?
Set that baseline before the rollout, not after. Retrofitting a baseline once adoption is universal makes the comparison impossible, and the pressure to declare success will be considerable by then.
“Measure the system, not the individual. The individual will tell you they are faster, and they will believe it.”
