Case Study: eBay improves A/B testing and performance reporting with CACI Limited’s Bayesian framework

A/B Testing For Improved Online Performance Reporting

eBay, the global online marketplace with more than 100 million active users, wanted to better understand how changes to its website affected visitor behaviour and revenue. To improve the speed and reliability of its A/B testing, eBay worked with CACI Limited on a framework for Bayesian significance testing that could provide deeper insight into customer behaviour.

CACI Limited developed and benchmarked a new Bayesian reporting framework for eBay that delivered additional daily metrics, clearer probability-based reporting, and more intuitive visual outputs. The solution helped eBay shorten or extend tests more flexibly, improve confidence in experiment results, and run more tests by using testing bandwidth more efficiently, supporting faster optimisation of the buyer and search experience.

eBay

Matt Gardner

Senior Manager EU Buyer and Search Analytics