When a patient walks into a hospital with chest pain, the doctor does not stop at recording the symptom. The chest pain is a symptom, but it is not a diagnosis. The cause could be acid reflux, anxiety, a heart attack, and so on. The symptom starts the investigation while the diagnosis determines the treatment.
Modern medicine can scale because healthcare does not rely only on narrative descriptions of symptoms. Instead, it classifies conditions, giving physicians, hospitals, researchers, payers, and regulators a shared language for understanding disease, comparing outcomes, tracking recurrence, and improving care.
The first publication of the International Classification of Diseases (ICD) by ISI more than 100 years ago made this possible. ICD did not replace clinical judgment; rather, it gave medicine a common diagnostic vocabulary. It also did not start as an exhaustive map of every possible medical condition (the first international revision in 1900 had only 179 categories of diseases and external causes of death such as disease of the respiratory system, disease of the nervous system, epidemic disease, etc.). The power was in creating a shared structure that could evolve.
The enterprise AI industry needs its ICD moment because complex systems cannot improve at scale if every failure remains an anecdote describing a symptom.
For the last two years, the industry has focused on the most visible symptom when analyzing failures, loosely termed “model hallucination.” But as ChatSee’s recent State of Enterprise AI Failures report indicated, less than 10% of failures can be categorized as a hallucination. Hallucinations still matter, but AI agents increasingly misfire in other ways: retrieving the wrong context, over-correcting, invoking the wrong tool, skipping a required escalation, or completing a task through a strategy the business would never have approved. So, given this diversity, enterprises should move from ad-hoc log and symptom processing to systematically categorizing diagnoses of failures so repeated remediations, best practices, and trends can be addressed consistently.
Symptoms Are Not Diagnoses
Most organizations already have some telemetry into their AI systems. They capture prompts, responses, traces, tool calls, latency, API errors, user ratings, and completion rates. That visibility is useful, but it is often closer to a symptom chart or intake metrics than a diagnosis.
A failed agent action is a symptom. But was it a tool-selection failure, an authorization failure, a schema mismatch, a missing dependency, or an invalid workflow step? A repeated retry is a symptom. But was the agent adapting appropriately, stuck in a loop, trying to work around a constraint, or failing to escalate?
A successful completion is also not proof of correctness. An agent can complete a task through an invalid strategy: bypassing a required review, using the wrong data scope, calling a tool it should not have used, or producing an outcome that satisfies the immediate request while violating enterprise policy.
This is why enterprise AI needs more than logs, traces, dashboards, and one-off evaluations. It needs the ability to ask what type of failure occurred, where it occurred in the AI lifecycle, what evidence supports that conclusion, how severe it was, and what treatment is appropriate.
Without a diagnosis, every incident becomes a story. With a diagnosis, incidents become patterns.
Why Agents Make The Solution Urgent
The need for diagnostic categories is not unique to AI. Security has vulnerability classes and attack techniques. Software engineering has defect taxonomies and root-cause analysis. Cloud operations have service-level indicators and postmortems. IT service management has ticket categories, severity levels, and resolution codes.
Those systems are not perfect. But they exist because complex systems eventually need a shared language for failure.
Traditional software generally follows paths engineers designed in advance. When it fails, the question is usually which code path, dependency, configuration, permission, or infrastructure component broke.
Agents introduce a different problem. They are given objectives, tools, context, and discretion over how to complete work. They can choose among actions, adapt when they encounter obstacles, call tools in different sequences, interpret policies, and sometimes discover strategies the operator did not anticipate. That means the failure may not be a broken component; it may be a bad strategy.
What an AI Failure Taxonomy Looks Like
When our team started building a failure taxonomy, we began with failure events rather than model benchmarks: public incidents, community-reported failures, open-source agent traces, Hugging Face datasets, AI interaction logs, production observations, and real enterprise workflows where AI systems did not behave as expected.
We analyzed more than 10,000 observed AI failure events, mapping failures into more than 150 categories across 10 major industry verticals, business functions, and seven lifecycle stages. The taxonomy includes chatbot-relevant failure categories, such as reasoning-related ones, including intent misclassification, logical consistency, and unsupported claims. In addition, many important categories become visible when AI systems are connected to tools, data, workflows, policies, and escalation paths (e.g., missed escalation, unresolved task, invalid workaround, and more).
This enabled the shift from “the agent gave a bad answer” to “the agent failed during the policy and governance stage by not escalating a fraud-sensitive complaint. The treatment is to strengthen escalation criteria and add workflow controls so fraud-sensitive cases route to human review before closing.” That’s a very different diagnosis and a very different fix.
From Diagnosis to Memory
The AI failure taxonomy is just the first step. Diagnosis requires constant learning from what happened so the same failure does not keep recurring.
That is another lesson from medicine. A diagnostic system does not only support treatment of an individual patient. It also supports population-level learning: which conditions are increasing, which interventions work, and where preventive action is needed.
Enterprise AI needs the same shift.
Today, many AI failures disappear after they are handled: a user complains, a ticket closes, a prompt gets adjusted. But the enterprise rarely builds a durable memory of the failure class itself. That means the same type of failure can reappear in another agent, another business unit, another workflow, or another vendor implementation.
Failure memory changes that. When an enterprise classifies failures consistently, it can recognize recurrence. It can detect when a new agent is showing an old pattern. It can recommend controls based on prior failures. It can turn incident review into reusable learning.
Medicine didn’t get its shared vocabulary because every disease was mapped from day one. It got there because clinicians stopped treating each case as a one-off story and started building a system that could recognize what it had seen before. It’s time for enterprises to take that same approach. AI agents will undoubtedly fail. Enterprises can either build the foundation to learn from each failure and recognize patterns to prevent them from happening again – or expect to keep rediscovering and diagnosing the same failure over and over.
Sekhar Sarukkai is the co-founder and CEO of ChatSee.ai, an AI failure intelligence platform. He previously co-founded Skyhigh Networks, acquired by McAfee; Securent, acquired by Cisco; and Confluent Software, acquired by Oracle.
