Case Study: Gusto achieves 3x AI support deflection with Humanloop

Gusto triples AI deflection rates with Humanloop

Gusto, a leading payroll and HR platform, faced the challenge of ensuring the accuracy and reliability of its AI-powered assistant, Gus. They needed a systematic way to evaluate AI performance, improve response accuracy, and efficiently iterate on prompts without causing regressions. To address this, they turned to the vendor Humanloop and implemented its AI evaluation system.

By integrating Humanloop into its CI/CD pipeline, Gusto automated evaluations against real-world data, blocking deployments that didn't meet quality thresholds. This allowed domain experts to refine AI responses and provided continuous monitoring. Using Humanloop, Gusto tripled its AI-powered case deflection rate from 10% to 30%, saving millions in support costs and enabling faster, more confident deployment of AI model improvements.

Gusto