Case Study: F500 Telecom improves virtual assistant customer satisfaction measurement with Snorkel AI

How an F500 telecom uses Snorkel AI to measure and improve virtual assistant CX

F500 Telecom, one of the largest telecommunications companies in the U.S., partnered with Snorkel AI to improve how it measures customer satisfaction in its virtual assistant conversations. The company struggled with sparse, skewed survey feedback and slow, costly manual reviews, which made it difficult to understand customer sentiment at both the conversation and individual message levels.

Snorkel AI helped build AI models for conversation-level satisfaction scoring and turn-level agent quality monitoring using SME-aligned LLM-as-a-judge, synthetic training data, and a lightweight Qwen-based reward model. The solution achieved a 67.7% SME alignment rate for the judge, 79 macro F1 on conversation rating, 81% accuracy in predicting turn-level satisfaction, and 7x faster evaluation, enabling real-time monitoring, better dashboards, and proactive intervention.