Most organisations are trying to control AI costs in the wrong place. They compare model prices, cap access and look for cheaper inference costs. Sure, these are still useful cost-reducing measures, but they often come into play too late. By the time an oversized prompt reaches a model, a duplicate dataset has been copied into another environment or an agent has queried half the company’s data estate to answer a narrow question, much of the cost has already been designed into the system.
The AI bill is often a downstream symptom of how data is prepared, accessed, governed and delivered. A leaner foundation determines how much information must be moved, stored, scanned and sent to a model every time it is used.
The scale of the problem is becoming clear. Gartner estimates that agentic AI models require between five and 30 times more tokens per task than a standard chatbot, and warns that falling token prices will not be fully passed on to enterprise customers. Every token is billed, and how many a task needs is decided by the data architecture underneath, not the model on top.
Stop feeding models the whole warehouse
Many generative AI services charge according to tokens, API calls or processing time. The FinOps Foundation has identified managing the cost and use of tokens as a leading challenge because spend can be difficult to allocate, forecast and connect to business value. I am not saying monitoring does not matter, it does, but it only tells an organisation what it has consumed. The larger saving comes from reducing what the model needs.
Retrieval systems are too often built on the assumption that more context produces a better answer. As a result, companies double down on feeding more content, but feeding pages of loosely related documents or broad table extracts into every prompt consumes tokens, adds processing and leaves the model to separate evidence from noise. For instance, a model asking about an overdue invoice does not need the customer’s entire history. It needs the relevant account, invoice, payment terms and approved business definitions.
That level of selectivity depends on how the data is prepared. Curated data products package a focused dataset with the metadata, ownership, lineage, quality measures and access rules an AI system needs to understand and use it correctly.
It is within this context layer that cost disciplines become actionable. When AI systems draw from curated data products – focused datasets packaged with metadata, ownership, lineage and access rules – organisations can trace which data sources are feeding which workloads and apply access policies that limit what any given agent or application can reach. That visibility makes it possible to connect consumption patterns to specific use cases, set access boundaries before unnecessary queries are made, and route requests to the right data rather than the broadest available set. The goal is not to monitorspend after the fact, but to reduce what the model needs before the first call is made.[Text Wrapping Break][Text Wrapping Break]Starburst AIDA puts this into practice. Rather than exposing a model to broad table scans or loosely related documents, AIDA answers business questions by drawing exclusively from governed, curated data – applying the access controls and business context defined by the organisation. The result is an AI interface that reduces what needs to be sent to a model by design, not as an afterthought.
Stop solving every data problem with another copy
AI projects have a habit of creating data copies. Records move from operational systems into a warehouse, then into a project environment, a vector database and testing stores. Each choice may be reasonable alone, but together, they create storage charges, duplicate pipelines, reconciliation work, and several versions of the truth.
Each new copy creates more maintenance and more opportunities for the data to fall out of sync. Teams end up repairing pipelines and reconciling conflicting records, while proprietary formats and tightly coupled systems make changing providers harder and more expensive.
Centralisation still has a role where performance or repeated reuse justifies moving data. It should not be the automatic first step. Federated access can let AI systems query selected data where it lives, while open table formats can make storage usable by different compute engines. Your teams need the discipline to decide what should move, what should remain in place, and what should be materialised, because the performance gain outweighs the duplication.
AI makes neglected data expensive
Open formats and federated access do not remove the cost of poor data maintenance. Small files, redundant snapshots and weak partitioning force systems to scan and process far more data than each query requires.
That may be manageable when a predictable set of dashboards runs a few times a day. It becomes far more expensive when AI applications and agents continuously query the same data in less predictable ways.
Compaction, retention policies, snapshot expiry, orphan-file cleanup and query optimisation therefore have a direct bearing on AI costs. When these tasks are neglected, every request carries unnecessary storage and compute overhead before the model has even been called. The aim is not simply to store less data, but to make sure AI systems can reach the right data without pulling unnecessary infrastructure into every interaction.
Governance belongs in the cost model
Governance is often treated as a compliance workstream that follows innovation. For AI, that separation is expensive. An organisation that cannot identify which data a model used, where it was processed or who was authorised to access it may have to redesign the system before production.
Good governance can also reduce consumption. The GDPR principle of data minimisation requires personal data to be adequate, relevant and limited to what is necessary. That is a legal principle and sound AI economics. The EU AI Act adds risk-based obligations for providers and deployers, making clear that controls and accountability are increasingly important.
Data sovereignty belongs at the same design stage. Organisations should know where data resides, which models can see it, whether it crosses jurisdictions and how access is logged. Keeping data in place where practical, applying policy at access and delivering only the approved slice can reduce movement costs and compliance exposure.
Leaders should ask how much data a use case needs, whether the system receives trusted context, why another copy is being created and whether consumption can be traced to a result. AI cost control begins before the first model call. The model may generate the answer, but the data architecture determines what it costs.
