Application security’s new blind spot
Application security spent a decade learning to see inside our software, building tools like software composition analysis, the software bill of materials, and dependency graphs to know what we actually ship. For open-source components, it worked.
Then artificial intelligence entered the codebase. Models, agents, and the connective tissue between them now shape how applications behave, and most organizations cannot say which are running, where they came from, or who approved them. It is the central blind spot in how software gets built in 2026, and most teams haven’t named it yet.
AI is now the development environment
Treating AI as an ’emerging’ technology has become a liability. Checkmarx’s 2026 Future of Application Security report, which surveyed 2,350 developers, AppSec managers, and CISOs across 14 countries, found that roughly half of production code is now AI-generated, and that 96% of organizations have AI components embedded in or connected to their applications.
Human-written, first-party code is no longer the norm. When most of what reaches production is written by a machine or pulled from open source, the foundations application security was built on: visibility, ownership, accountability, control, no longer hold.
Yet governance hasn’t kept pace. The same report found only about a fifth of organizations have a formal process for governing the AI components in their applications. The environment changed completely; the guardrails didn’t.
Shadow AI is the new shadow IT
Every prior wave of technology adoption produced a shadow version, unsanctioned tools entering the organization faster than anyone could track them. AI is following the same pattern, but with higher stakes, because the unsanctioned tool is now writing and shaping production code.
The signal is already visible to the people closest to the work. In the same study, AppSec managers reported unauthorized AI coding activity at rates three to four times higher than executives or developers perceived it. Research from MIT’s Project NANDA has similarly found that employees across the vast majority of companies use personal AI tools for work regardless of policy.
The lesson of shadow IT applies directly: you cannot govern what you cannot see, and prohibition without visibility simply drives usage underground. The first step in governing AI is not writing a policy. It is building an accurate inventory of what is actually running.
The SBOM can’t see AI
The software bill of materials is one of the most important advances in supply-chain security of the past decade. It gives organizations a structured, machine-readable inventory of the software packages they depend on, and it has become a baseline expectation for regulators and customers alike.
But the SBOM was built to track packages, not intelligence. Consider what’s entering codebases now: a model fine-tuned on a company’s own data, an agent that makes its own decisions while the software runs, or an MCP server handing an AI access to other tools. None of these fit neatly on a dependency list. They behave differently every time, they raise hard questions about where they came from and how they’re licensed, and they introduce risks a package inventory was never built to catch: data leaking out, prompts being manipulated, an agent acting without authorization.
This is why a parallel discipline is now forming around the AI Bill of Materials, or AI-BOM. Work led through the OWASP community aims to standardize inventories of AI models, datasets, and dependencies in much the same way SBOMs standardized open-source visibility. The goal is not to reinvent governance, but to extend the discipline we already trust to a layer of the stack that currently has none.
Regulation is about to make this unavoidable
The pressure will not stay internal for long. Emerging requirements are converging on a single expectation: organizations must be able to account for the AI in their systems. The EU AI Act sets out technical documentation and transparency obligations. The NIST AI Risk Management Framework and ISO/IEC 42001 establish governance baselines. The EU Cyber Resilience Act extends supply-chain accountability further still.
The common thread is documentation that holds up under scrutiny. When an auditor, a customer, or a regulator asks which AI models are running and where they originated, “we’re not sure” will not be an acceptable answer. The organizations that struggle most will be the ones that treated AI inventory as a future problem rather than a present obligation.
Crucially, audit-ready evidence has to be deterministic. A probabilistic guess about what might be running, expressed as a confidence score, is not the same as a finding that traces back to a specific file and line of source code. Governance becomes real only when the evidence behind it is defensible.
The deeper problem: seeing is not the same as acting
Here is the finding from that 14-country research that should unsettle the industry most. The problem is no longer detection. It is execution.
Across the survey, 73% of organizations rated their security posture as advanced or highly mature, yet 93% reported a breach in the past year tied to a vulnerable application they built themselves. The organizations rating themselves most mature reported the highest volumes of AI-generated code and the highest rates of knowingly shipping vulnerable code.
This is the pattern that should reframe how we think about AI visibility. Adding an inventory of AI components is necessary, but it is not sufficient. If the industry repeats its existing habit, generating more insight and then failing to act on it, an AI-BOM becomes one more dashboard that documents risk without reducing it.
What needs to change
Prevent. Embed security at the moment of code creation through an AI agent operating inside the development environment, whether that’s an AI-native IDE (Cursor, Cognition, AWS Kiro) or an agentic coding tool like Claude Code. Security has to be present as the AI assistant generates code, not bolted on after the commit, so insecure patterns are caught at the point of lowest intervention cost.
Discover. Make visibility continuous rather than periodic through an AI Bill of Materials (AI-BOM) backed by model, MCP, and agent scanning. AI components change with every commit, and an inventory that’s accurate once a quarter is inaccurate most of the time — so discovery has to live inside the pipeline and the developer’s workflow, not in a review held after the fact.
Remediate. Bring agentic triage and remediation to the pull request (GitHub) and to the existing backlog of security vulnerabilities, resolving risk where developers already work rather than in a separate queue. This depends on the highest-fidelity scanning results available: maximizing true positives, measured by F1 score, so engineers spend their time on real, exploitable findings instead of triaging noise.
Orchestrate. Tie it together with application security posture management (ASPM) and risk orchestration into a governed, repeatable process. Visibility, approval, and enforcement only hold when coordinated across the pipeline and backed by an audit trail, producing the documentation a regulator will ask for before they ask for it.
