The Bill Arrives After Launch: What an AI Feature Really Costs to Run

An AI feature is the first kind of software most companies have shipped that costs more the better it does. Every previous feature had a build cost and then, mostly, sat there. A feature built on a language model charges by the use, and if users like it, the invoice climbs with them. Teams that budget it like the old kind of feature find this out in the second month. 

I run a software agency, and we build these features for clients and run them inside our own products. The build is not where the surprises live. They live after launch, in three places that are easy to name and easy to skip: the meter, the drift and the outage. Here is what each one does to a product, and the decisions that make them survivable. 

The Meter Runs on Success 

The first surprise is arithmetic. In testing, a feature is used a few hundred times by people who know what it does. In production it is used by everyone, in ways nobody planned, and each use is a paid call to a model. The feature that cost pocket money in staging is now a line on the monthly bill that moves with adoption. 

This is not a bug in the feature. It is the feature working. But it changes how it should be designed. The questions that matter are the ones that reduce spend per use without reducing value: which requests can be answered from a cache because they repeat, which can be handled by a smaller and cheaper model because they are simple, how much of the context being sent with every request is actually needed for the answer. On Sourcd.ai, an outreach assistant we built for a recruiting client, the growing cost with usage was one of the two things that most shaped the engineering around the model. None of that work is visible in a demo, and all of it decides whether the feature can be left switched on. 

Quality Drifts Without Anyone Touching It 

The second surprise is that a feature which worked at launch can get worse with no change to its code. The model behind it is a service, and the provider updates it. Real users send inputs the team never tested. The answers shift, subtly, and the first sign is usually a complaint rather than an alert. 

The only defence is an evaluation set: a collection of real inputs with known good outputs, run every time anything changes, including things the team did not change. It does not need to be large. A few dozen cases drawn from real usage, kept current as the feature grows, is enough to notice a shift within a day of a provider update instead of a month after it. It sounds like overhead. It is the cheapest part of the whole operation, because it converts a vague feeling that “it seems worse lately” into a list of failing cases that an engineer can fix. Teams that skip it are not saving money. They are deferring the discovery to their customers. 

Providers Go Down, and Limits Bite 

The third surprise arrives on an ordinary afternoon. The provider has an outage, or the product hits a rate limit it never approached in testing, and the feature stops. If the feature was bolted on with no thought about failure, the user sees an error where a result used to be, and the support queue fills. 

The decision to make before launch is what the feature does when the model is unavailable. Sometimes it is a second provider behind an abstraction, so the product can switch. Sometimes it is the old manual path, kept alive for exactly this. Sometimes it is an honest message and a retry. What it cannot be is nothing. On the same Sourcd.ai project, the provider’s limits and outages were the other force that shaped the build, and the fallback we had to design was as much a part of the feature as the prompt. 

The Cost That Depends on What You Chose to Build 

How large these three costs become depends on a decision made much earlier: whether the feature integrates an existing model or builds something new. The two paths carry different running costs and different risks, and the choice is often made by accident. The fuller comparison of AI integration vs AI development is worth reading before a roadmap commits to either, because the answer changes what “after launch” looks like. 

For most products, integration is the right call, and the running costs above are the price of it. Building a custom model trades some of them for others: less dependence on a provider, far more dependence on the team’s own data, training and upkeep. Neither is free. The mistake is choosing one without knowing what it costs to run. 

Where the Client’s Money Actually Goes 

When a company asks what an AI feature will cost, the honest answer has two parts, and the second is the one that surprises people. 

The first part is the build, and even that is not what most expect. Clients tend to assume the feature is a plug-in, arriving cheap and fast. The real work is data access, evaluation and guardrails: getting the model the information it needs, proving its answers are good, and deciding what it must never do. The model call itself is a small share. 

The second part is the running cost described above, which never ends. A monthly bill for inference. Engineering time to keep the evaluation set current and to investigate drift. A fallback that has to be maintained even when it is never used. A feature with no owner after launch is a feature that will be switched off within a year, usually after an expensive month. 

Three Decisions Before the First User 

None of this is a reason not to build. It is a reason to decide three things before launch instead of after. 

•   A cost per use you are willing to pay, and a way to measure it. If the number is unknown, the first month of real adoption will supply it, at a price.
•   An evaluation set, however small. Twenty real cases with known good answers, run on every change, catch most of what customers would otherwise report.
•   A fallback that is a product decision, not an engineering afterthought. What the user sees when the model is gone should be designed, not defaulted.

A team that has answered those three can put an AI feature into production and leave it there. A team that has not will still ship, and will then spend the following quarter discovering the answers one invoice, one complaint and one outage at a time. 

Author bio:

Artemii Tkachuk is the founder and CEO of IvorySoft, a software development agency that has built mobile, web and AI products for startups and small businesses since 2018. The company also builds and runs its own products, ThauRed and My Baby is Hero.