
AI’s potential is well documented. But it comes at a cost. Organisations face rising bills from model providers, due to uncontrolled AI use, a lack of governance, and poorly-defined use cases.
AI has the potential to change the way we work, driving efficiencies and productivity gains on a scale that can be compared to a new Industrial Revolution.
But the gains promised by AI are by no means certain. As many as 90% of AI projects fail to make it beyond the proof-of-concept phase. Organisations using AI, especially “frontier” models, face rising costs.
As a result, businesses are struggling to produce a return on investment from their AI initiatives. Unmanaged, consumption-based AI use raises the prospect of out of control spending.
As AI vendors look to recoup their own costs, the price of AI tokens is rising. Initiatives to use AI as much as possible, or “tokenmaxxing”, look increasingly unsustainable, at least for the majority of businesses.
Rising costs
Organisations face increasing bills for AI use, especially from cloud-based frontier models. Token costs are rising. Initiatives to promote AI use can lead to spending running out of control. And yet, only a minority of organisations report a significant ROI from AI projects.
This is by no means solely the fault of the AI industry. AI labs face their own capacity constraints and rising costs, not least for hardware and energy. They need to cover those expenses, fund the next generation of model development, and recoup their investment in, often free, public access to their tools.
But enterprises also face AI sprawl, with an ever-larger set of applications including AI functionality or making API calls to AI models. These features can come with hidden usage costs.
Then there is the challenge of shadow AI. Individuals, or teams, might have their own AI tools outside the organisation’s policies.
As with cloud computing, and shadow IT before it, even organisations with robust, centralised technology frameworks are seeing users turn to their own, unofficial AI tools. Aside from the costs, this raises security, privacy, data sovereignty and business continuity risks.
Smaller models
One factor driving rising AI costs is enterprises’ over reliance on frontier AI models from the large AI labs.
These models are highly capable, and can tackle a vast range of business tasks. However, they are not cheap. If a business uses a frontier model for routine AI workloads, they are paying for capabilities they might not need.
Smaller models can carry out many AI tasks equally well, and costs can be as little as one tenth, or less, of the token costs of leading-edge models. And open-weight, open source models are increasingly powerful.
Some enterprises, too, will want to run their own AI models on in-house architecture.
Several factors drive this: data security, technology sovereignty, including local control of both the hardware and software, and wanting to use private, proprietary data sources for AI inference, and potentially, training.
Much of enterprises’ most valuable, and useful, data is also sensitive. Organisations will, understandably, be wary of sharing those data with a third party in the cloud.
In addition, uncontrolled access to external, frontier models can lead to higher than necessary AI fees. Organisations risk splitting their AI spending across a range of labs and tools, missing the benefits of volume-based, enterprise discounts.
Some boards will understandably prefer the certainty that comes with running technology, including AI, in house.
AI hardware is not cheap, but more efficient models are allowing organisations to use less expensive, more readily available equipment rather than scarce, high-end GPUs. And even for GPU-heavy workloads, it can pay to run AI locally. The up-front cost should be compared with ongoing, and rising, token fees.
To do this, enterprises need a way of achieving fine-grained control over AI, but without creating friction, or compromising ease of use.
Enterprises also need to prepare for agentic AI technology. Agentic systems can have far-reaching autonomy, creating their own pathways through business processes and calling on a range of AI models to complete the tasks. But without controls, this risks becoming very expensive, as well as hard to control.
The solution is to invest in tooling that monitors, and controls, AI use in the background.
Simple controls
Tools such as an AI Gateway provide both visibility of AI usage within the business, and help organisations control costs. This includes routing requests to an organisation’s preferred AI environment, based on policy and the nature of the prompt.
A task could be routed to an open source or open-weight model running on local hardware, to a smaller language model or an industry-specific AI provider, or, if it is justified, to a frontier model.
The advantage is that this happens in the background, and is transparent to the user. The organisation has the flexibility to set, and update, their AI usage rules as costs and business priorities change, and as new, innovative models come online.
The gateway also improves visibility of AI use within organisations, and provides vital insights for policy development and evaluating use cases. This is all the more important as enterprises look to introduce agentic AI networks, with their own demand on model usage, as well as their own potential ROI.


