Case Study: Microsoft boosts Copilot retention by 5% with Outset

Microsoft boosts Copilot retention by 5% with Outset

Microsoft, a major technology company, faced a challenge in evaluating its conversational AI product, Copilot, due to the highly variable and personalized nature of every user interaction. Traditional small-sample UX research and AI benchmarks were insufficient to capture the diversity of real-world experiences at the necessary scale. To solve this, they partnered with the vendor Outset to implement a new research methodology.

Using Outset's AI-moderated interview platform, Microsoft conducted large-scale "UX evals," capturing both quantitative ratings and qualitative feedback from thousands of users. This approach provided deep insights into user preferences and identified gaps in the product experience. As a result, Microsoft was able to make targeted improvements to Copilot that led to a 5% increase in user retention, a significant impact for a product of its scale.

Microsoft