Case Study: Upwork improves chatbot accuracy and cuts QA time with MaestroQA

Upwork cuts chatbot QA time from 16 hours to seconds with MaestroQA

Upwork, a large online marketplace, faced challenges with its customer support chatbot, including inaccurate responses and hallucinations. Their manual quality assurance process was inefficient, taking over 16 hours per week to review only a tiny fraction of conversations. To address this, they turned to the vendor MaestroQA and its Chatbot QA product to implement a structured evaluation system.

By using MaestroQA's automated and AI-assisted tools, including AutoQA and Custom Rubrics, Upwork created a scalable solution. This implementation delivered significant results: QA time was reduced from 16 hours per week to mere seconds, AI accuracy improved, and monthly coaching sessions increased by 300%. The vendor MaestroQA enabled Upwork to proactively refine its chatbot, leading to a 30% increase in customer satisfaction and a more efficient support operation.

Upwork