The traditional shift-left model was built around the assumption that engineers read every line of code to understand changes and validate them before anything ever ships to production. But AI agents are now generating code much faster than engineering teams can ever hope to review it, which means that teams who adapt successfully are completely moving away from traditional diff reviews. Instead, these teams are monitoring high-level agent conversations and spinning up multiple development agents in parallel to accelerate their output. And they are discovering that while the code produced by these tools is syntactically clean and passes basic linting, it frequently introduces incorrect architectural decisions that spread invisibly through a codebase and keep returning even after you think you have completely removed them.
Mayank Bhola, who serves as the Co-Founder and Head of Products at TestMu AI, sees this exact organizational consequence play out consistently across enterprise teams that invest heavily in AI coding tools without investing equally in observability infrastructure. And he believes that framing this development shift purely as a testing problem is the completely wrong approach for modern engineering leaders to take. Bhola explains that a standard testing problem is easily solvable by adding more tests or simply running those tests earlier in the deployment pipeline. But AI-accelerated development actually creates a deep visibility problem that represents a gap between what the system is doing and what the team can actively see it doing. Because of this reality, engineering teams can solve this issue by building dedicated observability layers that make agent behavior visible at the exact layer where architectural decisions are actually being made.
When his team built KaneAI as an end-to-end testing agent, they realized the true challenge was not generating tests at a high speed. They focused instead on building a visibility layer that explicitly tells a team whether the tests check the right things and whether the validated behaviors correspond to the original business intent. Engineering leaders can replicate this success by creating tracing systems that log why an AI agent selected a specific code path rather than just validating the final code output.
Shift Your Focus From Testing to True Visibility
Diagnosing this issue as a visibility gap is far more precise because it properly locates the problem where the architectural choices are actually occurring. Graham Siener, who operates as the VP of Product at Honeycomb, identifies the specific mechanism that makes this visibility gap structurally different from older quality assurance problems. Siener observes that developers cannot write a test for a decision they did not even know was made by the system. And he points out that AI agents generate code today while making sweeping architectural decisions as unintended side effects of completing simple isolated tasks. Because these choices do not show up in a standard code diff, they are completely illegible to a human reviewer who is trying to move at the speed of automated development. To counter this risk, engineering managers should implement real-time tracking of agent prompts and contextual inputs so that every hidden assumption becomes a visible part of the review process. Otherwise, an incorrect architectural pattern gets established quietly and resists correction for months because every engineer who encounters it later assumes that someone else made a deliberate decision to put it there.
Traditional quality responses like adding more tests or enforcing high coverage thresholds fail because they do not touch the layer where this specific visibility problem lives. While a standard test validates code behavior against a strict written specification, a visibility problem means the specification itself is drifting in ways that no test can actively measure. This is not a problem unique to development teams. Security practitioners are encountering the same structural gap. Mahesh Kumar Goyal, Senior Data and AI Engineer at Google, argues in a recent conversation about Securing Agentic AI against data leakage with InfoSec Relations that existing governance frameworks built for static data and predictable systems cannot follow the dynamic, nondeterministic paths that agentic systems take — a visibility problem that maps directly onto what development teams are now discovering in their own codebases. But engineering organisations can solve this drifting specification problem by establishing continuous telemetry that maps codebase patterns against the real-time changes introduced by AI tools.
Bhola frames the product architecture consequences of this distinction in terms that engineering leaders building agentic systems will immediately recognize. He notes that the teams making the most progress are not the ones with the most comprehensive test suites. Instead, they are the ones that build deep observability directly into the decision-making layer of the agent so they can see exactly why a decision was made. Without that specific layer of visibility, a test suite only tells you whether outputs match a potentially outdated specification. At TestMu AI, this exact principle shapes how KaneAI treats agent decision points as first-class observability objects rather than as mysterious black boxes that either pass or fail. Engineering teams can apply this model by building dashboards that flag whenever an AI agent deviates from established organizational coding patterns.
Build Production Observability as the True Foundation
Inverting the conventional relationship between software testing and production monitoring represents a major structural shift for engineering teams dealing with autonomous code generation. Siener argues that teams must start with production observability as their clear foundation rather than leaving it as an afterthought at the end of the lifecycle. He suggests flipping the traditional model because most teams building software historically focused on testing first and added production observability much later. To put this into practice, teams can connect their AI coding tools directly to live production telemetry systems. This allows the autonomous agent to see exactly how similar code behaved in real production environments before it makes architectural decisions that are expensive to reverse. And this tight feedback loop compounds over time because every single incident diagnosed with good telemetry teaches the system something new about high-risk code paths and fragile configurations. Because the production behavior feeds back into the agent context before the next change is made, the system gets progressively better at avoiding bad decisions entirely.
The compounding nature of that feedback loop is what distinguishes it from conventional observability as a debugging tool. When production behavior feeds back into the agent decision-making context before the next change is made rather than after the next incident surfaces, the system gets progressively better at avoiding the class of decisions that produced previous failures rather than simply detecting them faster after the fact.
Martin Reynolds, who serves as the Field CTO at Harness, extends this production observability argument directly into the corporate governance layer to make it operationally sustainable at an enterprise scale. He explains that the most effective organizational shift involves moving away from trying to prove total correctness before a release toward proving the ability to detect and correct errors quickly after deployment. Reynolds notes that manual governance is what actually slows engineering teams down whereas codified and automated governance serves as the true enabler for rapid development. This automated model creates a safe permission structure where developers can move fast because the system actively enforces the architectural rules in real time. Engineering leaders can achieve this by embedding automated policy checks into their continuous integration pipelines to block any AI-generated code that violates compliance standards.
Design Custom Visibility for Specific Codebases
Meaningful visibility must start at the upstream code review layer long before any generated software ever reaches a live production environment. Sylvain Kalache, who leads AI Labs at Rootly, discovered that code review became a massive operational bottleneck because the volume of pull requests increased much faster than human capability. To solve this problem, his team built a custom AI reviewer using the Claude Agent SDK to read full code diffs and evaluate them against their unique engineering standards. Kalache explains that commercial AI reviewers serving thousands of companies optimize for low costs by using smaller models and compressed diffs which makes their suggestions generic by design. But his team wanted deep and highly accurate reviews that reflect the exact way they build software within their specific organization. Running at a low cost per review, this custom tool provides the precise visibility that generic development tooling simply cannot match. Engineering leaders can adopt this strategy by training internal models on their own historical repositories to ensure that automated reviews catch team-specific architectural drift.
Building visibility that is specific to how a team builds software rather than how software is built in general is the quality investment that closes the gap generic tooling leaves open. This matters because the architectural problems that spread through AI-accelerated codebases are team-specific rather than universal and require team-specific detection to surface.
Subrat Prasad, a Founding Engineer at Share.xyz and formerly a software engineer at Google, highlights the high stakes of this operational shift by sharing how his team rebuilt their entire quality infrastructure after a major development breakdown. Following a severe disruption that took two weeks to resolve, his team built a new workflow centered entirely around learning from actual software failures rather than relying on theoretical upfront design. Prasad explains that their testing coverage now grows organically from real failures because end-to-end tests provide genuine confidence that new changes do not silently break existing behavior. Of course, the clear tradeoff is that these end-to-end tests run more slowly on pull requests than traditional unit tests usually do. But this inverted testing pyramid acts as a highly effective visibility strategy that ensures the team can see whether core software behavior is holding up under pressure. Teams can implement this approach by prioritizing comprehensive end-to-end user flows over simple code coverage metrics to guarantee system stability.
Bhola concludes the visibility argument by noting that successful teams stop treating observability as something to add after a system is built and instead treat it as a foundational property from the very beginning. This mental shift changes what teams choose to instrument and transforms the engineering relationship with AI-generated code from reviewing static outputs to actively monitoring dynamic decisions. Under this model, quality assurance becomes a continuous operational signal rather than a rigid gateway that blocks development velocity. And the upfront investment in visibility determines whether AI-accelerated development compounds into a durable corporate capability or transforms into technical debt that takes years to untangle.
Moving from a rigid gateway model to a continuous signal framework represents the shared operational posture that multiple engineering pioneers have arrived at through completely different development paths. This integrated strategy is the only one that successfully holds up when code moves faster than any human review process was ever designed to absorb. And it allows modern engineering organizations to maintain absolute control over their software architecture while fully capitalizing on the immense speed of autonomous generation tools.
Siener summarizes this new industry reality plainly by stating that quality assurance used to be a simple gate that code passed through before deployment. But in an AI-native development workflow, it becomes a continuous signal that engineering teams must always read to maintain system health.
Author: Saqib Jan
Email: [email protected]
BIO: Saqib Jan is a technology analyst with experience in application development, FinOps, and cloud technologies, helping organizations accelerate modernization practices and enhance their security posture. He also contributes as a technical author to industry-leading publications and top IT outlets.
