The Metaverse. Google Glass. Second Life. The Segway.
History is littered with “next big things,” new tech that’s hyped as world-changing, civilization-altering, inevitable… only to fade from view once the noise dies down.
AI is the latest entry in this long tradition. AI is not a fad: it’s real, it’s useful, and it’s not going anywhere soon. But there is hype. There is a dramatically overstated vision being sold: that AI will replace human workers en masse, render entire professions obsolete, and fundamentally rewire civilization, all within just a few years. But from where I stand, this is about as realistic as it would have been to expect Ford’s 1908 Model T to lead to a world where there’s a flying car in every garage by 1920.
The Illusion of the Second Opinion
As an application security researcher, I see this as organizations race to embed AI into their software development lifecycles. AI tools are being adopted to help accelerate development and security processes alike, but they’re too often being given far too much trust. That unearned trust tends to create a false sense of security.
Research published by IEEE.org shows AI code assistants routinely miss subtle but dangerous issues: business logic flaws, authentication state mismanagement, multi-step injection chains. Their outputs seem confident: long explanations, authoritative tone, plausible reasoning. But that confidence is largely unearned. While there’s value in the capabilities, trusting an LLM’s pattern analysis as truth, or even as genuine expertise, is a big risk.
When an AI review step in the pipeline seems to give confident results, it becomes easy to assume deeper reviews aren’t needed. But that’s not an assumption the industry can afford to make. A 2024 study by Stanford and Google DeepMind researchers examining code generation models found security-relevant errors in roughly 40% of AI-generated code samples. Trusting those same systems, even if they have improved with more recent models, to be the primary assessor of code security just doesn’t make sense.
Dev teams believe the hype. But recent research found that AI-generated security feedback not only misses real problems and suggests fixes that don’t work; it can also waste developer time by confidently reporting issues that simply don’t exist, suggesting fixes that aren’t needed.
Given that 81% of AppSec leaders admitted their organizations already knowingly shipped vulnerable code, the last thing we need is more noise coupled with a false sense of security.
Why AI Misses What It Misses
Don’t get me wrong: AI tools excel at catching well-documented, frequently occurring vulnerabilities — SQL injection, hardcoded credentials, deprecated cryptographic functions, and the like. Yet they struggle with contextual and compositional vulnerabilities: issues that only emerge when code is used in a specific situation or when two individually benign logic paths interact to create something dangerous, like a race condition.
Giving the tools much more context improves matters, but also significantly increases the cost of using AI. More context means more tokens, more compute, more spend.
According to NIST’s Secure Software Development Framework, contextual threat analysis remains a human-centered discipline for the foreseeable future. Adversarial reasoning is improving with frontier models, creating both powerful automation for security teams and new threats as adversaries adopt the capability. But defense is harder to scale with the same capabilities and doing so effectively requires more than what LLMs can provide.
Enterprise Adoption Is Outpacing Maturity
Gartner found that AI-augmented tools now present in the majority of large enterprise security toolchains, adoption having roughly doubled in two years. The technology is spreading faster than the playbooks needed to use it safely.
Some organizations are buying the vision of AI security tooling, expecting it to significantly reduce security headcount, replace red teams, and cut security tool spend. But this is betting safety on a vision that hasn’t yet arrived and has a real chance of never arriving. AI security tooling does, if used intelligently, amplify human capability; it does not replace it. Drop AI review into an engineering culture that isn’t already security-minded, and it won’t increase security vigilance. It just makes people feel like it did.
Striking the Right Balance
This isn’t an argument against AI security tooling; it’s a plea to be clear-headed and data-driven in how you adopt it. Applied thoughtfully, these tools provide real value: scaling pattern-matching across enormous codebases, reducing triage load, and surfacing obvious issues early. McKinsey Global Institute research found AI assistance can reduce certain security defects by 20–30% when layered onto an already effective AppSec program.
That word — layered — is everything. These tools work as noise-reducers and triage accelerators, not oracles. They work when the humans receiving their output know enough to interrogate and be skeptical of them. They work when an AI is treated as the beginning of scrutiny, not its conclusion.
AI Security That Holds Up to the Hype
Cybersecurity has always longed for a silver bullet. But over 20 years in the industry has taught me to be very skeptical of anything claiming to be one. Firewalls were once thought of as fortresses. Automated security testing was going to end insecure software. The pattern is familiar: innovation, outsized promises, then painful recalibration when adversaries find the gaps.
AI security tooling is the latest chapter in the hype cycle of tech. There’s real value to be had, and it’s unwise to ignore it. But it’s also a Model T, not a flying car. AI tools help, they accelerate, they enhance. But they also miss important things and provide a false sense of security that erodes security cultures and critical safety behaviors.
The most effective security organizations of the next decade won’t be those that deployed the most AI. They’ll be the ones that deployed it wisely, with an honest assessment of its capabilities and limitations and its relationship with specialized security tools. They’ll be the ones with a deep appreciation of the way human judgement augments automation. They’ll be the ones that buy reality, not the hype.
