You will find that artificial intelligence is most often measured by how sophisticated its models are. In a product team or the boardroom, there is a tendency to put your faith in AI-assisted forecasting when the numbers look precise be it a risk estimate, a sales projection, a probability score or some model of customer behavior.
The trouble is, in the real world, an AI-backed decision can go awry for reasons that have nothing to do with a technically deficient algorithm. More often it is because those using it have framed the problem too tightly, are working from their own preconceptions, or simply put the wrong question to the machine.
Rebecca Surtay has been at pains to put this issue under a microscope. A B2B marketing and product development specialist hailing from Kazakhstan, she is well versed in human-centered AI and data-driven decisions. Having cut her teeth in corporate settings where managerial judgment and market entry strategy collide, she has come to focus on an urgent matter for any company making use of AI: what do you do when a perfectly sound model is fed into a biased process?
Surtay does not advocate turning your back on AI, only on being more disciplined in its application. To that end she has put together a validation protocol for AI forecasts. It is meant to flag the kind of distortion that can occur in a business environment due to overconfidence, poor prompt design or cognitive bias.
It is no small thing given the current climate. From healthcare to operations, firms are being pushed to bring AI on board and they do so with alacrity. But governance rarely keeps pace. A team will be all too ready to have an AI tool make a prediction, but far less inclined to wonder if the prompt has unduly influenced the result or if they are in a position to dispute an answer that tells them what they want to hear. That is the gap Her work focuses on, the human layer of validation that is so often left out.
When AI is right, but the decision is still wrong
When AI-assisted decision-making goes wrong, you will often hear it put down to a data issue. The model was not well trained, the prompt was wanting, the dataset had gaps or the tool simply wasn’t up to the task. While those are genuine risks, Surtay would say they are only part of the picture.
The more insidious kind of failure in a corporate environment tends to come after the model has done its work. You might have a forecast that is taken as objective fact even though it was born of a narrow line of questioning. Or an executive team will nod at a probability score because it gives them cover for the strategy they want to pursue. A recommendation can seem to carry the weight of data and be all the more credible for it, despite untested business assumptions.
That is where human bias makes its way into the AI process. Surtay singles out overconfidence and confirmation bias, as well as poor initial framing. In projects with an AI component, these are liable to be magnified. If a team is set on a particular product launch or client approach, they may shape their prompts and read the results in a manner that flatters their preconceptions. In effect the AI ceases to be a support tool and turns into a very sophisticated mirror.
To counter this, her methodology puts in place a protocol for managerial and product decisions. It is not enough for an AI forecast to be impressive; under her system you have to ask if it has been put to the test, contextualised and checked against outside views and similar cases.
The three-part validation protocol
At its core, Surtay’s framework is a blend of causal analysis, reference class forecasting and an independent review by stakeholders.
Take the causal analysis component for instance. It is there to make sure teams do not just accept correlation at face value. When an AI model tells you a project is likely to be a success, you have to probe the why. What are the factors the tool is counting on? Do they still hold true? You need to be on the lookout for any unspoken constraints in the market, with your customers or from regulators. In short, causal analysis stops you from taking the model’s output as gospel until you have grasped the logic underpinning it.
Then there is reference class forecasting. This means decision-makers should measure their project against comparable cases instead of fixating on the particulars of what is in front of them. In B2B you will often find people who think their work is one of a kind, but this element of the framework puts the evidence front and centre: how did similar efforts fare? What was the failure rate and why? Were any red flags overlooked?
The final piece is having an independent stakeholder put the AI’s forecasts through their paces. Under Surtay’s protocol, you don’t leave it to the project team alone; you bring in someone with no emotional stake in the result. That could be a domain expert, an outside adviser or an executive from another function. The idea is to foster some structured disagreement before you have put your time, capital or reputation on the line, not to stifle innovation.
Put these three together and you have a sensible way to govern decisions made with AI. There is no need to scrap your tools or put in place something overly complicated. The framework simply provides a process that can be repeated so you are asking the right questions before you act.
From B2B forecasting to international discussion
At her masterclass for the NextWave Awards international program, Surtay put forward an approach under the title “Why Perfect Forecasts Fail: Human Factors in AI-Assisted B2B Projects.” The audience was a mix of some 20 countries’ worth of professionals from as far afield as China and South Korea to the U.S., U.K. and Canada, not to mention India, Singapore and Turkey. You would have found B2B project heads, AI specialists, corporate and marketing types as well as academics in the room.
Surtay made a point of being practical with the material. Instead of indulging in some abstract discourse on artificial intelligence, she zeroed in on the ways AI-assisted forecasting can come undone in the course of actual business. She put forward instances where teams were too trusting of their projections and missed things like stakeholder conduct or the gap between what a model says is probable and commercial reality.
Take one of the B2B cases she went over: the AI gave an assessment with a high likelihood of success, yet the results told a different story. In Surtay’s view, you could not just say the model was at fault. The trouble was the team had not put its assumptions to the test; a combination of past wins and a certain internal confidence meant they let the AI confirm what they were already inclined to believe.
Then there was the matter of applying standardized KPIs in various markets. What looked like a sound plan on paper did not hold up when you got to the local level. It is a case in point for her argument that while an AI tool may be fast at churning through data, it has no innate grasp of market nuance or the human elements of execution.
Her message was not to have any misgivings about AI forecasts, but to make sure they are properly governed.
Why the framework matters beyond B2B marketing
You could say the reach of Surtay’s protocol is broader than the B2B projects that were the subject of her masterclass. The human-factor risks she outlines are just as much in evidence in fields like healthcare, education, wellness and preventive digital health.
Take BodyFusion for example. Tied to Surtay’s work in AI-assisted movement analysis and wellness, it is a project built on dance, intelligent motion analytics, visual feedback and the like. While it is not put forward as an alternative to clinical care, its value is in things like body awareness, coordination and making movement education more accessible.
In these kinds of settings one has to be judicious with AI. A system can spot patterns and offer guidance, but the user’s experience is contingent on good, human-centered design. Does the technology support the person or get in the way? Is the feedback something they can understand and find motivating, or does it chafe? Does the recommendation suit their condition?
At first blush you might not see what AI forecasting in business has in common with movement analysis. But the principle is the same: an AI’s output is only of use if you put it in the proper human context. Get the framing wrong in a healthcare or wellness application and the consequences can be severe. You risk missing emotional or accessibility considerations by defining success too narrowly, or alienating users by treating them as data points. An unvalidated system may work on paper but be useless in practice.
That is the import of Surtay’s human-factor approach, which applies well beyond corporate forecasting. It gives you a framework for viewing AI deployment as a matter of the relationship between the model, the decision at hand, the context and the user.
A practical answer to the overconfidence problem
There is an enduring danger in the way organizations are taking on AI: it can be a convenient cover for old habits of overconfidence. An executive with a lot of self-assurance has a model to point to these days; a thin strategy can be propped up with a well put-together forecast, and you can get what appears to be an objective answer from a question that was not well posed.
The protocol is intended to reduce this risk by introducing structured review before decisions are finalized. Through her framework, an organization has a means of putting the brakes on when it needs to by mandating independent review, reference class comparison and causal analysis. You could say it is there to keep teams from running headlong into unexamined decisions, but it does not stifle innovation in any broader sense.
That is no small matter for AI governance. While plenty of firms are drawing up policies on data protection and who gets to use their AI tools, not many have put in place the kind of practical routines needed to check an AI-influenced business call. Her work addresses a practical gap that many organizations still struggle to formalize: how to review AI-influenced business judgment before it becomes action..
And she is not coming at this from a place of technophobia. She understands that adopting AI is as much a socio-technical affair as anything else. The data and the model are important, certainly, but then you have to factor in the psychology of the user, the incentives at play, how people communicate and where decision rights lie.
Toward more accountable AI-assisted decisions
You will find AI woven into the fabric of management, product design and healthcare as well as digital wellness. Yet the next frontier is not a matter of churning out more powerful models; it is about building a sturdier decision architecture to support them.
Surtay’s work reflects a broader shift in AI governance toward the point where model output meets human judgment. With her framework, she recasts bias from some amorphous ethical worry into a tangible risk for business and product that can be put in check with the right methods. In doing so, her work speaks to a number of disciplines, from B2B strategy and enterprise AI governance to responsible AI, digital health and human-centered design. The common thread is this: the issue is never just if the AI can come up with an answer, but whether those in charge are equipped to judge it before they make a move.
Do not mistake the appeal of AI-assisted decisions for the idea that machines will banish uncertainty. They won’t. What you have is the prospect of better outcomes when an organization pairs computational might with a certain discipline in its human validation.
Her protocol is one example of how such validation can be structured. It would have companies put aside their awe at a model’s accuracy and ask a more grown-up question: have we put the decision itself to the test? When it comes to the current wave of AI adoption, that question is becoming increasingly relevant as AI moves from experimentation into everyday business use.
