Case Study: ALLEN Digital achieves 71% GPU cost savings with CAST AI

ALLEN Digital saves 71% with CAST AI

ALLEN Digital, an e-learning platform, faced rising costs and underutilized GPU instances with their AI models running on Amazon SageMaker. Seeking to maintain performance while reducing expenses, they turned to CAST AI and its Kimchi Inference solution to address their infrastructure inefficiency.

By implementing CAST AI's Kimchi Inference, ALLEN Digital leveraged GPU time-slicing to run multiple models on a single instance and utilized a mix of Spot and on-demand instances. This approach dramatically increased their GPU utilization and resulted in substantial cost savings of 71% compared to their previous SageMaker setup, all without compromising on latency or performance.

ALLEN Digital