Case Study: Shopify achieves high-quality product catalog taxonomy labeling with Toloka

Shopify achieves 95% taxonomy accuracy with Toloka

Shopify needed to accurately map millions of its merchants' user-generated products into a structured taxonomy with over 10,000 categories. The challenge was to process 22,000 active product collections within days while achieving a minimum of 95% accuracy, a volume and speed impossible for human-only annotation due to the noisy and complex nature of the data. They partnered with Toloka to find a solution.

Toloka implemented a sophisticated hybrid pipeline where AI models and human annotators worked together. The solution used an ensemble of vector-based and tree-search AI methods to handle the massive taxonomy, with trained Toloka annotators focusing on high-disagreement cases and multimodal gaps. This approach allowed Shopify to achieve its target of over 95% accuracy, ensuring high-quality ground truth data to power its global Catalog for AI and agentic commerce.

Shopify