Optimizing Customer Satisfaction Predictions in Call Centers

Customer Satisfaction (CSAT) is a big deal for call centers. It's like the gold star that shows how well they are doing. But here's the catch: only a small number of customers actually fill out a CSAT survey after their call. We're talking around 8% in some cases. This not-so-great response rate can make it hard for call centers to know how happy their customers really are. Missing out on Feedback means they might miss some key chances to improve their service.

To tackle this problem, call centers might want to use a model that predicts how satisfied a customer is, even if they don't fill out the survey. Since CSAT is so important, it’s vital to make sure these Predictions are as accurate as possible. This is where our research comes in. We have come up with a method to make sure that these predicted customer satisfaction scores, or pCSAT, closely match the actual survey results.

#Setting the Scene

It’s not uncommon for machine learning systems to get updated. The tricky part is that these updates can change the balance of results. For example, if too many predicted scores are high or low, it could throw off the overall picture. To fix this, we created a control process that helps keep these scores in check, especially when there's a lot of sampling noise (think of it like static on a radio).

In our findings, the average CSAT scores can vary wildly if not everyone responds. If only a small fraction of customers give feedback, what about the rest? Predicting satisfaction for all calls can help get a clearer picture.

#The Challenge of Customer Feedback

Let's face it, we all know surveys can feel tedious. When customers don’t respond, their opinions remain a mystery. Predicting customer satisfaction for every call could help smooth out these rough edges. Our paper offers a new way to predict these scores without introducing bias.

#What’s Out There?

In the world of machine learning, predicting customer satisfaction has gained quite a bit of attention. Studies have shown different ways to tackle this issue, but they often struggle with keeping the true distribution of survey results. We took a closer look at previous research to understand their methods and see where we can do better.

Some researchers have used automatic transcription systems to analyze call transcripts, along with non-text data, to create satisfaction scores. Others have looked at how acoustic features help predict satisfaction. Our approach builds on previous work, allowing us to improve the accuracy of predicting CSAT scores based on call transcripts.

#Replicating Class Distribution

The cool part of our method involves ensuring the predicted scores mimic the actual survey responses closely. We need to make sure different satisfaction levels are accurately represented, so no one feels left out.

In the world of machine learning, there are ways to handle imbalanced data. Techniques like re-sampling and adjusting thresholds can improve how classes are represented. However, these methods often don’t help much when it comes to getting an exact match to survey data. To get specific and useful predictions, we had to optimize decision thresholds. This means making accurate predictions while keeping the natural ordering of satisfaction levels intact.

#Building the Model

To create our predictions, we used a large language model (LLM) trained on call transcripts. This model provides binary outputs: high or low satisfaction. We then use probabilities from this model to output our pCSAT scores. By carefully setting decision thresholds, we can accurately translate these probabilities on a 1-5 scale.

Our product requirements are clear: ensure the average pCSAT aligns with the average survey CSAT. We don't want any wild discrepancies.

#Gathering Data

We relied on transcripts from our Automatic Speech Recognition engine, which boasts a solid accuracy rate. We analyzed around 892,000 calls with known satisfaction scores. To make sure we weren't just lucky, we ran our tests several times under different conditions.

We also made sure to exclude call centers with too few responses. This helps us avoid unnecessary errors caused by sampling noise and allows us to focus on centers with a good amount of feedback.

#The Magic of Threshold Optimization

Our model uses a mapping function that takes a low satisfaction probability as input and yields a score on a 1-5 scale. The mapping consists of decision thresholds that separate different satisfaction levels. By estimating these thresholds, we can find the sweet spot to minimize errors while balancing the needs of different call centers.

#How We Tested

We ran our model through different scenarios to see how well it performed. In the first couple of tests, we looked at the average satisfaction levels. After comparing our predictions with actual survey results, we saw where we could improve.

For centers with lots of responses, we noticed a trend: the more feedback we had, the more accurate our predictions became. This makes sense; less feedback means more noise, which can confuse predictions.

#Results and Observations

Overall, our tests revealed that the method we devised was effective in predicting customer satisfaction. The loss rates varied based on call centers' response volumes. It was clear that for centers with fewer responses, our model struggled more. Still, for centers with a decent amount of feedback, we achieved impressive results.

#Fine-Tuning for Different Centers

We learned that a hybrid approach could be beneficial. For call centers with fewer than 200 responses, we could use one method, while relying on another for larger centers. This strategy ensures that we're making the most accurate predictions possible, regardless of how many customers take the time to respond.

#Ethical Considerations

As we developed this method, ethics were at the forefront of our minds. We want to ensure fairness and transparency in our approach.

We actively consider bias in our predictions, using different methods to evaluate groups of users. Our commitment to transparency means we've clearly documented our processes and findings, helping everyone understand how we arrived at our results.

In the spirit of full disclosure, we follow strict data privacy regulations, ensuring that any customer data we use is anonymized. We also make sure to scrub any personal information to protect individual privacy.

#Conclusion

By improving our methods of predicting customer satisfaction in call centers, we aim to help businesses make better choices for coaching and follow-ups. This, in turn, leads to happier customers and better overall performance for call centers.

So next time you get a call and someone asks you to rate your satisfaction on a scale from 1 to 5, remember: that feedback matters, even if you don’t feel like filling out the survey. Your thoughts help shape how call centers can improve, making your next call a little bit better.

In the end, we’re just trying to keep all the customers smiling-not an easy task, but we’re here for it!