AI experiences succeed when users know how far to trust them. They help people judge when to rely, when to question and when to step in.
Trust between people usually has some emotional padding. It is shaped by history, relationship, reputation and accountability.
If a colleague makes a mistake, you might judge it in the context of their wider performance. If a friend lets you down, your view of them may change, but it is unlikely to make you distrust everyone else.
AI does not benefit from that kind of relationship. Trust is often based much more directly on observed performance.
Does this tool work? Did it get the answer right? Can I rely on it next time?
That makes AI trust more volatile. A strong experience can build confidence quickly, and a visible failure can damage it just as fast.
Recovery is not automatic either. When an AI system gets something wrong, users need some sign that the issue has been acknowledged and addressed. Silence can make the failure feel bigger because it leaves the user to fill in the gaps.
This is especially important when AI systems are presented as capable, intelligent or near human. The more confidently a product is positioned, the less forgiving users may be when its limits appear.
The additional worry is that distrust can spread. Recent research on “moral spillover” found that when one AI agent behaved badly, negative judgements carried across to other AI systems. The same effect disappeared for humans when individuals were given names, suggesting people do not judge AI and people by the same rules. One bad AI experience can end up tainting the whole category.
For businesses, trust must be considered in the design of new models and tools as they are implemented.
The real risk is misjudgement
Trust calibration is about matching a user’s level of trust to the system’s actual capability, the task and the stakes involved.
When trust is too high, automation bias can creep in. The user accepts the output because it came from the system, not because they have properly assessed it. They may skim past errors, ignore uncertainty or fail to apply the same scrutiny they would bring to a human colleague’s work. Research on automation bias has found that once initial trust is established, people are more likely to stop checking an AI’s work and notice only the evidence that confirms it.
That risk grows when people are already positive about AI. A recent study in Scientific Reports tested whether participants could judge if faces were real photographs or AI-generated.
Those receiving guidance labelled as AI made poorer decisions when they already had positive attitudes towards AI. Human guidance did not produce the same drop in performance. Enthusiasm, in other words, can lower people’s guard.
When trust is too low, disuse becomes the issue. The user ignores or overrides a system that is performing well because they are sceptical, confused by the interface or shaped by a previous bad experience. Earlier research found that people can abandon an algorithm after seeing it make an error, even when it remains more accurate than a human doing the same task.
This problem is harder to spot because it often leaves no obvious trace. A hallucination passed into a client document is visible. A useful AI recommendation quietly ignored is not.
For businesses building AI experiences, both failures matter. Over-trust can create reputational, operational or safety risks. Under-trust can stop good products gaining traction, even when the underlying technology works.
Good AI design should slow people down when it needs to
A better approach starts with clearer expectations. Users should understand what the AI can do before they discover its limits through trial and error. That does not mean overwhelming every interaction with disclaimers. It means giving people the right cues at the right moment.
The challenge is that every calibration cue also competes for attention. A confidence score, data-source label, or limitation disclosure can help users judge an output, but too many signals can make the interaction feel heavier than the task requires. Good design is not about showing every limitation, it is about revealing the right information at the right moment.
This is where public attitudes to AI become important. KPMG’s 2025 global study found that 54% of people are wary about trusting AI, with people often more confident in its technical ability than in its safety, security or wider societal impact.
In other words, people are not asking whether AI works. Many already believe it can. They are asking whether it works for them, whether they can rely on it to act responsibly, and in their interests. A system can appear technically capable while still failing to earn deeper trust.
For low-stakes tasks, the experience can stay light. For higher-stakes decisions, the interface should work harder.
It might show confidence levels, explain what data has shaped the answer, flag uncertainty or make space for review before action is taken. In those moments, friction can be helpful. It gives the user a reason to pause before accepting something too quickly.
Users should also be able to challenge or override AI outputs without feeling as if they are fighting the product. Disagreement should feel normal. If an AI recommendation is based on limited data, that should be visible. If a system is unsure, it should not disguise that uncertainty as confidence.
This is where many AI experiences still fall short. Too much product language is designed to build belief. It talks about intelligence, transformation and speed. Less attention is given to helping users build an accurate mental model of what the system is actually doing.
That may win attention at launch, but it is a risky way to build long-term trust.
Trust is built through accuracy and honesty
The aim is not to show users everything the AI does not know. That would make products slower, heavier and harder to use. The aim is to reveal enough for people to make better judgements.
A user does not need a technical breakdown of every model decision. They do need to know when an answer is uncertain, when the data is thin, when a recommendation is only directional, and when human review is sensible.
This kind of honesty can feel commercially uncomfortable. Product teams often want to reduce doubt, not introduce it. But clear limits can build stronger confidence over time.
Research by Marmolejo-Ramos and colleagues found that people who better understand how algorithms work do not simply distrust AI. They trust it more selectively, showing more scepticism in high-stakes situations while still trusting AI in familiar, everyday contexts.
Users who understand a system’s strengths and weaknesses are more likely to use it well. They are also less likely to feel misled when it falls short. In the long term, this can build more productive relationships with AI, and better engagement with new tools as they are rolled out.
The next phase of AI product design should be less focused on maximum trust and more focused on accurate trust. People do not need AI systems that sound certain all the time. They need systems that help them decide when certainty is justified.
