Case Study: Sangfor achieves 3x higher GPU utilization with CNCF-backed Kubernetes, Volcano, and HAMi

Sangfor boosts GPU utilization 3x with Cloud Native Computing Foundation

Sangfor, a leading enterprise cloud computing and cybersecurity vendor in China, faced challenges scaling its unified AI compute platform. Key bottlenecks included underutilized GPUs, complex resource governance in a multi-tenant environment, and the need to run both inference and batch workloads concurrently. To address this, they worked with the Cloud Native Computing Foundation ecosystem, implementing a solution built on Kubernetes, Volcano, and HAMi.

The solution involved using HAMi for GPU virtualization and sharing, which allowed multiple model inference services to share a single GPU with strong isolation. This was coordinated with Volcano for advanced job scheduling. The results were significant: GPU utilization increased by more than 3x, the number of models per GPU increased to 8x or more, and external model invocation costs were cut in half from 400K to 200K RMB per month. The Cloud Native Computing Foundation technologies helped Sangfor build a stable platform that now supports over 2,000 R&D engineers.

Sangfor