The Best AI Feature May Be Knowing What Not to Change

What 69,051 chat messages reveal about control, iteration and the decisions behind generative AI. 

Historical analysis: September–November 2025. Total messages are not the same as user prompts. 

Imagine asking an AI system to change a wall colour. It produces a beautiful image, but also replaces the sofa and moves a window. The picture may be more impressive; the comparison is now less useful, because you cannot tell what changing the wall alone would do. 

That hypothetical example captures the distinction between generating an attractive output and helping someone make a decision. I co-founded Paintit.ai, an AI interior design platform, after working in architecture and interiors. Studying its usage led me to a different product question: what must an AI system preserve, not just produce? 

What the sample actually measures 

Our internal behavioural reports examined 69,051 total chat messages from 1,240 users across 5,134 sessions. The engagement report identifies the period as 1 September–17 November 2025. The wider product audience exceeded 120,000 users, but that separate platform-level figure is not the sample behind these findings. 

Messages, user prompts, sessions and people are different units. The reports distinguish total messages from user prompts; the intent percentages below describe extracted patterns in prompts, not percentages of the entire user base. They are descriptive internal findings, not an independently audited or representative study of all AI use. 

The request is small; the decision may not be 

The reported intent frequencies were add, 27.9%; replace, 20.1%; repaint, 11.3%; remove, 6.4%; and full redesign, 4.3%. These are selected extracted categories, not percentages of people or a complete distribution that should sum to 100%. 

The defensible finding is that specific modifications appeared much more often than an explicit full-redesign request. It would be wrong to translate 4.3% into ‘only 4.3% wanted creativity’ or ‘only 4.3% of users started from scratch’. A replacement can be creative, and a substantial project can be expressed through dozens of small instructions. 

The useful product lesson is narrower: an existing situation matters. People bring possessions, preferences and constraints into the interaction. Changing one thing without disturbing everything else is a meaningful capability, not merely a smaller version of generation. 

Paint is a test of whether context survives 

For Paintit.ai users comparing paints in an AI paint color visualizer, the image is an intermediate step in a physical decision. Walls appeared in approximately 25% of prompts, while repainting appeared in 11.3%. The reports also recorded 234 mentions of Sherwin-Williams, 224 of Farrow & Ball and 186 of Benjamin Moore: specific product references, not just abstract colour preferences. 

Those mentions do not establish purchases, brand partnerships or accurate colour matching. They show that some users brought real-world references into a generative workflow. The system then has two responsibilities: interpret the requested change and avoid inventing certainty about its physical result. 

A controlled paint comparison should hold the room, furniture and other relevant conditions steady while changing the requested surface. Otherwise, the user is comparing several variables at once. Sherwin-Williams’ own colour-visualisation guidance cautions that screen colours can differ from actual paint and recommends checking a physical sample. 

The general AI lesson is that making an option visible is not the same as validating it. A useful product should help people narrow alternatives, identify what remains uncertain and verify the decision outside the interface when necessary. Confidence is valuable only when it is justified. 

The second prompt matters as much as the first 

The median session contained four user prompts, and the mean was 6.9. In the same report, 31.2% of sessions contained four to six prompts, while 14.6% contained eleven or more. Only 17.3% contained a single prompt. 

That is evidence of repeated interaction, not automatic evidence of satisfaction. A long session could mean productive exploration, repeated repair of mistakes, or both. The report does not separate those explanations, so treating every additional prompt as engagement success would be misleading. 

Consider an illustrative sequence: change the wall, keep the floor, restore the original sofa. The later instructions carry information about what the user considers non-negotiable. An AI product that forgets them makes the user spend effort rebuilding context instead of progressing toward an outcome. 

This is not a new human–computer interaction problem. Microsoft Research’s Guidelines for Human-AI Interaction recommend supporting correction, remembering recent interactions and adapting cautiously. Generative interfaces make those principles particularly tangible: users need to revise an answer without surrendering the decisions already made. 

Figure 1. User prompts per session in the internal engagement report (5,134 sessions; percentages rounded). Interaction depth does not establish satisfaction. 

Segment by the job, not an imaginary average user 

A second internal report grouped the same dataset by project scope and contextual keywords. Approximately 15% of users were classified as working on whole-home projects, yet they accounted for 43.9% of sessions. Commercial or professional contexts represented 8.2% of users and approximately 29.7% of prompts. Together, whole-home and commercial jobs represented roughly 23% of users but about 71% of prompts. 

These are inferred job categories, not verified occupations. An ‘office’ could be someone’s own business; a mention of a ‘client’ suggests a different relationship. The analysis is useful for identifying workflows worth investigating, but it cannot reliably tell us every user’s profession or purchasing authority. 

A separate engagement classification identified 7.1% of users as power users, averaging 216.3 prompts each. That is a different segmentation axis, not a group to add to the commercial percentage. Together, the analyses suggest that a small number of demanding jobs can account for a disproportionate amount of activity. 

Subscription status reinforced the distinction: approximately half of whole-home users and 46% of commercial-context users were labelled Premium, compared with 30.9% overall. These are subscription shares within the snapshot, not measured conversion rates or proof that larger projects caused payment. Still, they offer a more useful product hypothesis than assuming everyone needs the same workflow. 

Another small signal came from household context: roughly 3–4% of users explicitly mentioned children, nurseries or babies, and around 45% of that group were labelled Premium. The report links these terms to focused, emotionally meaningful projects. The sensible follow-up is to ask users about their goal—not to infer a family situation or purchasing intent from a keyword and treat it as fact. 

In Paintit.ai’s AI room design workflows, a quick experiment may only need speed. For a multi-room or client project, saved decisions, version history, consistent constraints and a clear handoff become more plausible priorities. That is a reason to investigate the job behind usage, not to declare a professional segment from keywords alone. 

Do not mistake activity for an outcome 

The engagement report found that 347 of the 1,240 users, or 28%, returned on a different day during the observation window. This is not a standard day-seven retention measure, nor evidence that everybody else abandoned an unfinished task. Some jobs may be completed in one sitting. 

The reports also identify gaps: no reliable link from product references to purchases, limited long-term retention visibility, and inferred rather than consistently declared project goals. We therefore cannot claim that these interactions reduced returns, increased sales or prevented expensive mistakes. Those remain outcomes to measure. 

Even style language can mislead product teams. A keyword analysis found luxury-related terms in 22% of Trial prompts but only 7% of Premium prompts. That association is not a budget survey: people’s descriptive vocabulary and their subscription behaviour are different signals. 

Build for decision compression 

I use ‘decision compression’ as a working description, not a validated metric: reducing the effort between an unclear problem and a decision the user can evaluate. It is not about rushing people into a purchase or replacing their judgement. It is about preserving context while making the next meaningful choice easier to inspect. 

This also separates assistance from autonomy. Generating an option, recommending an option and acting on that recommendation are different steps. A user asking for a comparison has not necessarily delegated the final choice, and the interface should make that boundary visible. 

First, distinguish requested changes from invariants. A useful interface lets people specify both what should change and what must stay fixed, then carries those constraints forward. When the goal is ambiguous, a clarification can be more useful than another confident output. 

Second, make iteration inspectable. Show what changed, retain earlier versions and make reversal straightforward. A product should help a user compare alternatives, rather than quietly changing the basis of the comparison with every generation. 

Third, separate visual plausibility from external truth. Product references, measurements and paint colours need appropriate verification; a convincing image is not evidence that a real item fits or matches. The same principle applies whenever generated content is used to inform action. 

Finally, measure the path to an outcome. Candidate measures include unwanted-change frequency, time spent repairing outputs, whether users preserve a preferred version, and whether they report reaching a decision. These are proposed evaluation targets, not results our existing reports have already established. 

A useful hypothesis, not a universal verdict 

This is a single-product, historical sample shaped by its interface and by the people who chose to use it. Keyword extraction can miss intent or misclassify it, and chat records alone cannot establish how much uncertainty a person felt. We have not tested whether these patterns generalise to other AI categories. 

But the evidence does challenge a product habit: judging success mainly by the first impressive answer. In workflows with existing assets and real consequences, preserving what matters can be as important as generating what is missing. Better models and better control are complementary, not competing goals. 

The next useful AI product may still create extraordinary things. Its more important achievement may be knowing what not to change—and helping the user understand why the next step is worth taking. 

Sources and disclosure 

Internal data: How Users Design & Behave in Paintit.ai (November 2025), especially pp. 2–5, 15, 17, 19 and 22; Jobs & Hidden Segments (December 2025), especially pp. 6, 9, 11, 15, 18 and 19. Platform-scale context comes from a separate audience summary. The internal reports are author-supplied and not peer-reviewed. 

Context. Microsoft Research: Guidelines for Human-AI Interaction (2019). Sherwin-Williams: Paint Color Visualizer, including its physical-sample guidance. Links appear in the article. 

Related coverage. Business of Home previously covered the design-industry angle. This article develops the implications for AI product design, with explicit limitations.