How High-Quality Data in Generative AI Models Drives Customer Service Results, With Nga Phan of Salesforce AI

In early 2023, when the world was just uncovering generative AI, Clara Shih, CEO of Salesforce AI, discussed “3 Ways Generative AI Will Reshape Customer Service.” Since then, it’s been a consensus within the tech community that generative AI will transform customer service in countless ways. 

With high-quality data, generative AI can offer personalized and efficient customer experiences, reflecting an understanding of individual needs and preferences. The synergy between high-quality data and generative AI is revolutionizing the way customer service is delivered, marking a new era where intelligently automated yet human-like interactions between an organization and its customers are possible.

Nga Phan, SVP of Product Management at Salesforce AI, joined us at IMPACT 2023 to share why she thinks generative AI is so powerful for customer service, plus some best practices to help your data team begin driving value with Gen AI.

AI is the solution to customer expectations

AI isn’t a new concept. So, why are we seeing so much generative AI hype now? Nga believes it starts at the customer level.

Before joining the Salesforce AI team, Nga was leading the core service team for Salesforce Service Cloud. That meant she was focused on all things customer service-related: agent console and experience, case management, knowledge management, service automation, self service, and everything in between. Within this role, Nga gained a deep understanding of organizations’ desire to both meet customer expectations and improve agent productivity.

“We all know that customer service is strategic for a company to drive top-line growth through satisfaction, experience, and loyalty. But, customer expectations for receiving service have changed a lot in the last few years,” says Nga. “Customers expect service on their own terms – when they want it and how they want it.”

“It has been very difficult for companies to meet the rising needs of the end customers,” says Nga. “I think generative AI is a way to meet these rising needs at scale.”

For Nga, generative AI has the potential to help businesses deliver the type of personalized service customers are looking for in a more cost-effective way. 

“I believe that generative AI can benefit all of the industries out there,” says Nga. “In every business, there are lists of customer inquiries and requests that are routine and common, and you want to be able to respond to those in a conversational and human-like manner that’s targeted and personalized – but you don’t always need a human behind those responses.”

But start with the use cases and knowledge

There are several boxes to check to ensure generative AI is a good fit for the customer service use cases your business might be considering. Before an organization can launch GenAI models to improve customer service efficiency, it has to begin with customer needs.

“First, you need to understand the types of use cases you want to address with GenAI,” says Nga. “As a product team, you should always start with customer needs. Is our solution today solving their needs in a high-value manner? If not, how can GenAI improve our solution?” 

In evaluating what a GenAI model might look like, your team might ask questions like:

• What are routine and common requests that are high in volume?
• What kind of data would we need to enable agents or end-customers answer these questions seamlessly & efficiently?
• How should the end user interact with the GenAI solution? Does the user want the solution in their workflow? (For example, does a service agent want an AI solution to be where they’re doing their work, or do they want a copilot in a separate box?)
• When would the end users want the GenAI solution to be proactive in providing recommendations and assistance?

Think through the use cases, how the end user would actually consume the solution, and then evaluate available knowledge resources. Where there’s adequate knowledge coverage for high-volume service topics, there’s higher potential for Gen AI.

“It’s essential to have high-quality knowledge data,” says Nga. “You need to make sure the LLM has something to ground its answers on. You don’t want the LLM to use its general knowledge, which might not be accurate for your company.” She recommends pointing the LLM you use to your business’ set of high-quality knowledge data as a key early step in building out your models.

“You can leverage a general LLM, like GPT 3.5 or GPT 4, and unlock what it can do for you through prompt engineering with your enterprise knowledge data,” says Nga. “Or, you can take a general model or open source model like Llama 2, and use your own data to fine-tune the model for a specific task.”

With the right knowledge coverage and a thoughtful training strategy (RAG can be an excellent moat for customer service products), AI has the potential to supercharge all kinds of customer service use cases. 

Now, let’s dive into a couple of those use cases in a bit more detail. 

Generative AI for customer service: Use cases and examples

1. External case resolution: Customer-facing chatbots and assistants

A customer-facing chatbot is a great use case for GenAI, especially if your business typically receives many of the same routine questions from customers. But, to really be successful, your team also needs a robust knowledge source to provide the most accurate answers to those questions.

“Perhaps, up until today, it’s been hard to build a ‘human-like’ chatbot that could identify the intent of the customer accurately to extract the right knowledge for a particular question,” says Nga. Now, GenAI might have just enough “thinking” power to bridge the gap.

Nga proposes a chatbot solution that can function as a “Knowledge Bot.” This chatbot would mine all the articles on your website and bring the relevant information front and center in a conversation with a customer, removing the need for the end customer to search around your website (making it a much more pleasant experience) or a human agent to intervene on a low-impact case.

“Questions like ‘What is your return policy?’, ‘Can I still return a jacket I bought two months ago?’, or ‘Can I ship internationally for free as a Platinum member?’ can be easily answered by a Knowledge Bot,” says Nga.

2. Internal tools: Agent assistance and productivity tools

Gen AI can also have a significant impact on the agent side as well. A customer service agent’s work doesn’t begin and end with a customer email or conversation. A significant amount of manual work goes into properly preparing for a case (for example, researching the customer’s business), resolving a case (e.g., gathering the data artifacts, executing resolution), and closing a case (e.g., writing a summary of the conversation, adding information to a knowledge article), and more. And all of these actions take a tremendous amount of time and effort that could be better spent adding new value elsewhere (e.g., building relationship with the customer, suggesting additional products or services)

“Having the GenAI tool to summarize a particular case at the end of a conversation can be very helpful,” says Nga. “It can add the resolution or next steps. And if there isn’t an existing Knowledge Article that addresses the inquiry response, GenAI can help an agent generate a draft article with the push of a button.”

And that’s just one example. An agent could also leverage assistance via GenAI in another way: an AI-powered copilot. “An agent might say ‘I don’t see any knowledge articles for this customer case. Can you look at all the closed cases for a similar customer inquiry and help me summarize a resolution that might be useful for this case?” 

A use case like this would be incredibly tedious to resolve manually, but with GenAI, finding the right answer is as simple as asking the right question. There are many other powerful use cases for how a Copilot can assist an agent with issue resolution and it’s worthwhile to do an exercise to map out all the use case candidates against potential value and ease of execution to focus first on what brings the most immediate value.

But to get any of this value for your AI models, your model first needs access to one very important resource…

High-quality data.

The role of data quality in LLMs

AI is useless without the right data to power it. If the data is out-of-date or incorrect, the AI model is too. 

Of course, it might still make for a flashy presentation to the board, but you can bet it won’t be delivering any value to your customers and employees any time soon.

“You need to make sure you have the right customer data to craft the right prompt so the answer is personalized to the end user asking the question,” says Nga. “And you want to make sure the knowledge data that’s grounding the LLM is accurate and fresh.”

Monitoring your data with a data observability solution like Monte Carlo is the most scalable and efficient way to ensure high-quality data from the start of your generative AI journey onward. No one wants their GenAI model to source its answers from out-of-date or incorrect source data.

With Monte Carlo, your team will always be the first to know and the first to respond when bad data hits your models, improving time to resolution and decreasing data downtime as you build for reliable and trusted AI.

“Having high-quality, consistent data and a way to manage the data that’s feeding the LLM is the biggest challenge for many businesses. You need to solve that before you even start your generative AI journey.”

Trusted data means trusted AI.

To learn more about Nga Phan’s work at Salesforce AI, reach out to her here. And, to learn more about how your team can gain more visibility into your data as you begin a generative AI journey, contact our team. 

Our promise: we will show you the product.