What Enterprise AI Gets Wrong About Context

Context might be the most overused and abused term in enterprise AI right now. To make matters worse, there isn’t agreement on what is meant by the term. Ask ten vendors, and you’ll get ten different answers that probably each solve a different piece of the same puzzle. The part most organisations are working to solve is a big piece: defining how information interrelates, what things belong to each other, how concepts connect. But that top-down view of the world is only half of what an agent actually needs. The other half is the bottom-up work of organising an enterprise’s data estate so an agent can find the right information, understand how it is structured and be able to use it without going rogue. Value comes when the two meet in the middle, bringing together a definition of how the world works with a representation of the information an enterprise actually has. 

Good data structure is at the bottom context layer 

An agent needs guidance to make sense of an enterprise’s piles of files, raw tables and other sources of information. Organising data into data products can provide those directions, helping an agent understand where information is and what it needs to answer a question. They describe what information exists within a source, what it can and cannot answer, how it is organised, and how it relates to other sources. The specificity is what allows an agent to arrive at the right query, rather than searching broadly. 

This matters because the more work an agent has to do to find an answer, the more steps it takes and the more irrelevant material it draws into view, and both raise the probability of error. It also means more time, tokens and money are spent getting to an accurate answer. The objective should therefore be to give an agent an understanding of exactly what query to run, what information to use and how that information should be shaped. 

The top context layer defines how the world should work 

An agent needs guidance to make sense of an enterprise’s piles of files, raw tables and other sources of information. Organising data into data products can provide those directions, helping an agent understand where information is and what it needs to answer a question. They describe what information exists within a source, what it can and cannot answer, how it is organised, and how it relates to other sources. The specificity is what allows an agent to arrive at the right query, rather than searching broadly. 

This matters because the more work an agent has to do to find an answer, the more steps it takes and the more irrelevant material it draws into view, and both raise the probability of error. It also means more time, tokens and money are spent getting to an accurate answer. The objective should therefore be to give an agent an understanding of exactly what query to run, what information to use and how that information should be shaped. 

Governing the middle ground  

When the bottom layer and the top layer come together, a company has something an agent can actually work with: organised information mapped against a framework for how that information interrelates. 

But that combination also creates a governance problem. Once multiple individuals and business units are drawing on the same context layer, the question of who can see what, and what an agent can know on behalf of a particular person, becomes harder to get right. Access is determined by policy: who is asking, how the data is classified, and what that combination permits. Provenance and lineage are central to making that work, carrying a classification into everything derived from a source and establishing afterwards how information reached an agent. 

This is part of why governance has now become a popular word, after the industry spent years treating it as something that implied slowness. Yet sometimes you slow down to go fast. You can drive a racing car extremely fast precisely because it has brakes, and enterprise AI needs the same principle of control. 

That control needs to cover the actions an agent can take, the data it receives, how much history it can access, what it can know about other agents and people, how long it can run and how many tokens it can spend. Moreover, as agents are non-deterministic, those limits have to be enforced outside the model rather than left to its judgment, while guideline objectives, evaluation and feedback loops continuously assess whether an agent is meeting expectations. 

Why human knowledge still matters 

Organisations that have invested in curating data products with clear origin and structure are better positioned to manage permissions, because that classification travels with the data rather than being reconstructed after the fact. But provenance only accounts for information that was captured in the first place. A great deal of enterprise knowledge never makes it into a database or document at all. 

People hold knowledge about how businesses operate, why decisions are made and what different outcomes mean. The industry largely assumes that knowledge finds its way into the system, so that agents can draw on it. A good deal of effort goes into capturing it, but almost all of that produces prose for a person to read rather than something an agent can use. 

This matters because the dimensions of information an agent needs go beyond “what happened.” When it happened, who it happened to, and why it happened are equally important. Causality, which is understanding what drives what, is particularly hard to encode but meaningful for accuracy. Organisations that can begin to capture causal relationships, and encode them into the top context layer, give their agents an advantage in reasoning. 

The building is the hard part 

AI does not arrive knowing how your business works, where your information lives or what your people know. The data estate has to be organised so an agent can find the right information without searching broadly and hoping. The relationships between that information have to be defined so an agent can reason without starting from scratch every time. The knowledge that lives in people rather than systems has to be captured somehow. And all of it has to be governed so that the right people have access to the right information at the right time. 

The industry has been debating the word, when the real question is whether organisations are doing the work.