In Research, AI Gets You the What, but the Why and the How Still Come From a Person

Our clients come to us with questions they can’t answer from their own data. “Do people want this thing we’re building?” “Why did they choose our competitor over us?” In both cases, what the client is really asking is, “Do the people we want buying from us still care about our brand?” I run a research, analytics and AI firm, and for most of my career, finding those answers meant sitting in front of people and asking them questions. 

Some of that asking is done through bots now. The product team at a major technology platform hired us while they were updating their billing software. They wanted to know whether customers preferred it inside the big ERP system or as a standalone product. We ran sixty of those interviews with an AI moderator, and according to the summary, most people wanted it integrated. 

I wasn’t convinced. I had conducted some of the interviews myself, and that was not what I had heard from customers. I went back through all sixty interviews, one at a time, and found an even split: half wanted the software integrated and half wanted it as a standalone product. 

A fifty-fifty split had been reported as a clear preference, which was not true. As a researcher, you begin with a fuzzy picture that becomes clearer as you do more work, although it never becomes perfectly clear. AI can add some of that clarity; depth and texture depend on knowing what to listen for and what to check before you believe the result. 

AI is most useful when you need to conduct research at scale. You can run sixty or a hundred interviews at once, whereas I spend an hour sitting in front of one person. It also handles short-answer questions well, including “Have you used this?” and “Which brands did you consider?” The same applies to work you repeat, because a study you plan to run again next quarter is worth automating. 

When someone types their answers into a bot, however, we lose all the vocal and visual signals, including the emphasis on a word and the pause before an answer that tells me someone is hesitating. When they speak to the bot, we can hear their voice, but we still cannot see their face. When I’m sitting in front of someone, I can see the wince. That’s my cue to say, “Tell me more about that. You said one thing, and my sense was that you felt something else about it.” What they say next is usually an insight. Because a bot cannot register the cue, it doesn’t do a follow-up like a human would. I can also interrupt gently to clarify what I meant or ask someone to go deeper. When a bot cuts across someone mid-sentence to do the same thing, people often read it as rude and close up. I use AI for direct questions and leave the why and the how to the human-to-human discussion. 

I check the data before I check the insight built on it. When AI produces a summary, I put the data on one screen and the summary on the other, then feed it questions whose answers I already know. The test I use most involves giving it the wrong number. If I know the correct figure is 50%, I tell it the number should be 75%. When it comes back with 75%, I stop because I am reading my own input back to myself instead of the data. I push once to see whether it identifies the actual error or responds with, “Oh, you’re so right.” Giving it a wrong number tells me which kind of system I am dealing with. 

I used to spend whole days on some of the parts of my job. AI now handles work that can be completed without my intervention, and I do not miss it. Yet, within my organization and many others, we are asking the question of whether we are remaining relevant to our clients. Do they still need us? I have clients in the transportation industry, and when they explain what they do, they say, “I’m the last mile.” A system can move freight across the country, but someone still has to carry the parcel to your door. 

Research works in much the same way now. The technology can organize the findings and produce the analysis, but only a person can turn that analysis into an OhGee moment, our name for the aha moment, by explaining it clearly enough that a client knows what action to take. That is how the people running an organization can learn what their customers want and how to keep their brand useful to them. 

Aksel Bedikyan is the Founder & CEO of OhGee Insights, an award-winning data, AI and research firm serving the world’s largest enterprises. In 2022, he was named to ESOMAR’s Insight250, a global list of the most influential leaders in research, insights, and data-driven marketing.