Improving Customer Service with Language Models

Customer service is essential for many businesses, and companies often look for ways to improve this service while saving money. One method is using language models. These models can help human agents by suggesting responses to customer inquiries. This helps agents work faster and more efficiently, which is crucial because hiring agents can be costly.

Language models can automatically generate responses for agents, making it easier for them to respond to customers. However, while these models can be helpful, they also come with high costs related to training and using them. This article explores how businesses can evaluate the cost-effectiveness of these models and their impact on customer service.

#The Role of Human Agents in Customer Service

Even with advancements in technology, human agents remain important in providing customer service. Many conversations can be automated, but some situations require a human touch. Agents must handle multiple conversations and often need access to customer accounts and brand policies.

Due to the expense of employing agents, companies are seeking efficient solutions to reduce costs while maintaining the quality of service. One solution involves using systems that suggest responses for agents to use during customer interactions.

#Overview of Conversation Assist

LivePerson developed a tool called Conversation Assist, which helps agents by suggesting responses based on previous customer interactions. This tool speeds up the response process and allows agents to focus on more complex inquiries.

By using Conversation Assist, companies can expect reduced response times and improved response quality. A well-designed model can provide consistent and well-structured suggestions, even outperforming less experienced agents. This can lead to significant cost savings and increased customer satisfaction.

#The Costs and Benefits of Large Language Models (LLMs)

Large Language Models (LLMs) show great promise for customer service applications. They can generate high-quality responses, but training and operating these models can be expensive. For instance, using a specific model may cost more than others, which influences the overall expenditure for a business.

Moreover, the financial landscape around LLMs is constantly changing. Different companies may have unique agreements that differ from standard pricing. As technology evolves, the costs associated with training and serving LLMs will also change.

To help businesses determine whether using an LLM is worthwhile, a framework called Expected Net Cost Savings (ENCS) is proposed. This framework considers the financial savings from an agent utilizing a model's response minus the cost of generating that response. This approach can be applied on a message-by-message basis or in aggregate.

#The Case Study

To apply and test the ENCS framework, a case study was conducted with one brand. The focus was on using various methods to customize LLMs and see how they impacted costs and agent efficiency. Feedback from customer service agents was collected to evaluate three strategies: Fine-tuning, Prompt Engineering, and knowledge distillation.

The case study revealed that the usefulness of a response can significantly outweigh differences in the cost of generating these responses. This finding can be applied broadly to other businesses.

#Automation and Human Interaction

As businesses implement more automated solutions in customer service, human agents will still play a key role. While automated systems can handle simple tasks and general inquiries, they may not always adequately address complex customer concerns. Human agents can provide the necessary support when automated systems fall short.

The goal for most organizations is to strike a balance between automation and human intervention. By leveraging tools like Conversation Assist, companies make it possible for agents to respond quickly while ensuring customer inquiries are resolved effectively.

#Methods of Customizing Language Models

The case study focused on three methods to adapt LLMs for the brand in question:

#Fine-tuning

Fine-tuning involves training an existing model on a dataset specific to the brand. This helps the model to understand the language and context that is unique to that brand, resulting in better, more tailored responses.

#Prompt Engineering

Prompt engineering creates specific instructions or examples for the model to follow when generating responses. This method can guide the model in producing more relevant and accurate suggestions while reducing the need for extensive training.

#Knowledge Distillation

Knowledge distillation simplifies larger models into smaller, more efficient versions. This can help reduce the cost of running the model while maintaining a level of performance that is still useful for agent support.

#Evaluating Response Usability

To gauge the effectiveness of model suggestions, real customer service agents were asked to evaluate the usability of responses from different models. They indicated whether they would use, edit, or ignore the generated responses based on their experiences.

This evaluation led to valuable insights regarding which model configurations worked best in practice. By focusing on usability, businesses can ensure that the technology they implement meets the needs of their agents and customers.

#Analyzing Performance Metrics

A variety of metrics were used to assess the quality of responses generated by the models. These typically included factors like sensibleness, specificity, and helpfulness. These measures provided an overview of how well each model performed based on the agents' feedback.

Higher-quality responses, as indicated by these metrics, led to increased usage rates by agents. This relationship highlights the importance of not just focusing on the cost of generating responses, but also considering the actual value and usefulness those responses bring to customer interactions.

#Expected Net Cost Savings (ENCS)

The ENCS framework incorporates multiple factors, including model performance, agent cost, and the costs associated with generating responses. This approach allows for a clearer picture of potential savings and helps in making informed decisions about which models to implement.

By calculating how much time an agent saves when using a model’s response, organizations can see the financial benefits more clearly. The framework can also account for variations in different brands and their specific needs, making it flexible for various applications.

#The Future of LLMs in Customer Service

The case study suggests that LLMs have significant potential in improving customer service while providing cost savings. However, the landscape is changing rapidly. As new models are developed and costs fluctuate, businesses must remain adaptable and prepared to reassess their strategies regularly.

Both in-house and third-party solutions have their pros and cons. For smaller brands, the investment in in-house models may not be feasible compared to the flexibility that third-party services provide. However, for larger enterprises, the ability to control data and service quality might justify the upfront costs.

#Ethical Considerations

While technology can enhance customer service, it’s important to consider the ethical implications of using such tools. Ensuring that customer data remains private is crucial, and companies must be transparent about how they use these models.

Additionally, while the intention is to improve efficiency and service quality, there is potential that automation could lead to workforce reductions. This impact on human agents must be acknowledged and addressed to maintain a balanced approach.

#Limitations and Assumptions

The study has some limitations, including the sample size of agents and the potential biases in their feedback. Agent behavior in controlled evaluations may differ from real-life scenarios, which could affect how often they use model-generated responses.

Other factors, such as the complexity of customer inquiries or agent training, might influence the overall effectiveness of using language models. Future studies should explore these areas further to create a more comprehensive understanding of the utility of LLMs in customer service contexts.

#Conclusion

The integration of large language models in customer service can lead to enhanced efficiency and significant cost savings. By using techniques like fine-tuning, prompt engineering, and knowledge distillation, businesses can adapt these technologies to their specific needs.

The expected net cost savings framework provides a valuable outline for evaluating the financial impact of implementing LLMs in customer service. Companies must remain mindful of the evolving landscape and ethical considerations while leveraging these advancements to improve service quality.

Moving forward, organizations should prioritize usability and effectiveness when selecting language models, ensuring that the technology aligns with the needs of both agents and customers. By doing so, they can maximize the benefits of automation without compromising the human touch essential for exceptional customer service.