Case Study: Shopify achieves 96% lower serving costs and faster, higher-quality AI with PyTorch and vLLM

Shopify cuts serving costs 96% with PyTorch

Shopify, a global commerce company, faced the challenge of high costs and latency when scaling its AI features powered by frontier large language models. These general-purpose models were expensive to serve and, crucially, could not learn from production failures to improve their own performance. To address this, Shopify implemented a continual learning loop built with PyTorch.

Using PyTorch as its training foundation, Shopify built a system that automatically captures and learns from production failures. This solution, which also utilizes vLLM for inference, compresses this knowledge into model weights through daily fine-tuning and reinforcement learning. The results were significant: the fine-tuned model surpassed frontier-model quality, reduced latency by 38%, and cut serving costs by 96%, saving an estimated $26 million annually.

Shopify