Chatbots have been used now for many years to answer customer queries online in a scalable way, increasing employee productivity and customer satisfaction. However recent advances in generative AI have changed the chatbot game.
Large language models (LLMs) are a cornerstone of generative AI. They have reinvigorated chatbots with a more inherent and fluid understanding of questions asked, able to respond more naturally, making them an ideal interface for a broad range of users.
There is a rush to implement AI and reap the rewards much trumpeted in the press. Thus far that has taken shape in the leveraging of generalised AI models like ChatGPT and Bard. But is there a better route to success in the AI gold rush?
The specialised route to success
Short answer: yes. Rather than following the more generalised route, businesses should consider implementing specialised AI chatbots instead.
Why? While general chatbots like ChatGPT offer some level of utility and novelty for both businesses and consumers, harnessing general internet knowledge, they are not trained on private company data and therefore cannot answer specific business questions.
Specialised chatbots feature a general-purpose LLM but provide the LLM with specialised knowledge from which to formulate its response to a question culled from internal repositories.
The primary reason for specialised chatbots is that they can answer questions about a knowledge base that is internal to a company. For instance, contextual questions such as:
- How many weeks of paternity leave can I take after my child is born?
- What team developed the security features for the internal sales portal?
- How do I request a company AMEX?
… could all be answered by a chatbot with the ability to connect to the company intranet, file shares, and engineering wiki; a general chatbot would not be able to access this information, and would therefore be unable to provide this information to users across the business.
How to generate real ROI from your chatbot strategy
There are two approaches to getting an LLM to “know” about your internal company data: one is to fine-tune the LLM on your content, and the other is retrieval augmented generation (RAG).
Fine-tuning effectively takes a general-purpose LLM as a base and trains it on additional internal content so that it “knows” internal information. This process tends to be expensive both in terms of the hardware required to run the computations for fine-tuning, and the management effort to curate the data set and feed to the LLM.
This approach also has a “freshness date” problem in that any new information generated within the company’s repositories won’t be known to the LLM unless it is fine-tuned again. Fine-tuning also lacks any form of access control enforcement. Any information fed to the LLM becomes potentially public to all users.
The RAG approach on the other hand is much easier to maintain. There is no need to run a computationally expensive training process. Access control is applied when the information is queried so that the LLM is only presented with information that the user posing the question has access to, regardless of data location and file type. Because information retrieval and context building happen at the time that the question is posed, the information used to formulate the response is always up-to-date and secure.
Documents relevant to the user’s natural language question and appropriate to their level of access are retrieved from an index, including vector and then supplied as context to an LLM. This allows the LLM to respond to the user’s question with a factual answer based on contextual knowledge.
With specialised AI chatbots, businesses can increase employee productivity and reduce frustration for customers by getting them the answers they need, when they need them, with the minimum amount of input.
Getting an exceptional ROI from the deployment of AI chatbots therefore relies on unlocking the vast repositories of data most enterprises hold, including all the unstructured content, by using the RAG approach.
From RAG to riches: the future of chatbots
The RAG approach has the potential to give organisations a lift-off when it comes to maximising chatbot effectiveness. While the full impact of AI on a broader, societal level will continue to evolve over the coming months and years, businesses can start putting in place their foundations for success now. AI has been integral to the delivery of chatbots for many years, but recent advantages allow for chatbot strategies to be finessed via RAG.
This ability to specialise chatbots allows for a more contextually refined service for your chatbot users, whether that means employees or end customers. Ultimately, being able to uplevel the user experience in this way can enable organisations to steal a march on the competition in the AI gold rush to come.
