Your Product Doesn’t Need an AI Rebuild

Most requests to “add AI” arrive in the same shape. A founder or a product lead has a working piece of software, paying customers, and a growing sense that the product now looks old next to everything with a chat window. The request is rarely specific. It is a feeling that something has to change, and a worry that the something is the whole product. 

It almost never is. In the conversations my agency has had since language models became a line item on every roadmap, the right answer has usually been one feature, placed carefully inside software that already works, and the wrong answer has usually been a rebuild. This is what those conversations tend to reveal, and what the actual work looks like when the feature is chosen well. 

What “Add AI” Usually Means 

The first job is translation. When someone says they want AI in their product, they typically mean one of four things, and each leads somewhere different. 

Some ask for a chatbot. Pressed on what the bot should do, the answer is usually that users cannot find things: a setting, a document, an answer that exists somewhere in the help centre. What they need is search that understands a question, or a way to get an answer grounded in their own documents. A general-purpose chat assistant is the most expensive way to deliver that and the least predictable. 

Some are being pushed from outside. Investors ask what the AI story is. A competitor’s home page grew the word “intelligent.” The request has no user behind it yet, and the first task is finding one, because a feature built to satisfy a pitch deck will be judged by users who never read it. 

Many actually need automation. Somewhere in the product there is a step where a human reads something, decides what it is, and moves it along: an incoming request, a form, a support ticket. A model can extract, classify, route or draft. It is one step in a workflow, not the product, and the moment the team sees it that way the scope shrinks to something buildable. 

And nearly all of them expect it to be cheap and instant, a plug-in that arrives on a Tuesday. The model call is the easy part. The work is in giving the model the right data, finding out whether its answers are good, and deciding what happens when they are not. 

Why the Rebuild Instinct Is Wrong 

A rebuild feels like the honest option. If the product was designed before any of this existed, surely it should be redesigned around it. 

But the product’s value was never its architecture. It was the workflow it encodes, the data it has accumulated, and the habits of the people who use it. All three are exactly what an AI feature needs in order to be useful. A model that can see a customer’s history, act inside the screen where the work already happens, and hand its output to the same next step the user already knows is worth far more than a cleverer model in an empty new product. 

There is a quieter reason too. A rebuild delays the moment anyone learns anything. The feature that ships inside the existing product in a few weeks produces facts: whether users touch it, what they ask, where it fails. A rebuild produces a launch date. 

Retrofitting, Done Properly 

Adding a language model feature to software that already exists is a specific discipline, and it has a small number of parts that keep coming up. 

Data access before model choice. The feature is only as good as what the model can see. Before any prompt is written, the question is what the model needs from the product, and whether that data is reachable, clean and permitted. This is unglamorous integration work, and it is most of the budget. 

A provider you can swap. The model behind the feature will change, either because a better one appears or because the current one becomes unavailable for an afternoon. Putting an abstraction between the product and the provider costs a little at the start and saves a rewrite later. 

Evaluation from day one. A small set of real inputs with known good outputs, run every time something changes. Without it, quality drifts silently and the first person to notice is a customer. 

A fallback that is not an error page. When the model is slow, wrong or down, the feature needs a graceful path: the old manual step, a cached answer, a plain message. Deciding this at the start is far cheaper than after the first outage. 

This is the shape of the work involved in integrating AI into an existing product, and it is also why the estimate for “one feature” is rarely as small as people expect. None of these parts is optional. They are what makes the difference between a demo and something a business can rely on. 

What Production Teaches 

Whatever a team believes about an AI feature before launch, production corrects it. Three lessons have repeated across the products we have built and run. 

Cost grows with usage, not with features. A feature that costs almost nothing in testing can become a noticeable bill once real users adopt it, and adoption is the goal. Caching, shorter prompts and smaller models for simpler cases are not optimisations to add later. They are part of the design. 

Quality drifts. Provider updates change behaviour, and real users send inputs no test anticipated. The evaluation set is what turns “it feels worse lately” into something a team can act on. 

Providers go down and rate limits bite. On Sourcd.ai, an OpenAI-powered outreach assistant we built for a recruiting client as a Chrome extension, both the growing cost and the provider’s limits and outages shaped what we had to build around the model. The feature itself was the smaller part. 

When the Answer Is No 

Part of adding AI well is declining to. There are two situations where I tell a client not to build the feature at all. 

The first is when a rule, a filter or ordinary search would do the job. If the decision can be written down as logic, a language model adds cost and unpredictability for nothing. 

The second is when a wrong answer is expensive. Legal, medical and financial outputs are the obvious cases. One confident mistake can cost more than the feature will ever earn, and a model’s confidence is not a measure of its accuracy. 

In both cases the client usually arrives wanting the feature. The most useful thing an engineering partner can do is explain why they should keep the money. 

One Feature, Inside What Works 

The products that get real value from language models in the next few years will mostly not be new products. They will be existing ones with one well-chosen capability added where the data already lives and the users already are, built with a swappable provider, an evaluation set and a fallback, and measured from the first week. 

That is a smaller ambition than a rebuild. It is also the one that ships. 

Author bio:

Artemii Tkachuk is the founder and CEO of IvorySoft, a software development agency that has built mobile, web and AI products for startups and small businesses since 2018. The company also builds and runs its own products, ThauRed and My Baby is Hero.