Case Study: China Merchants Bank boosts AI compute efficiency with CNCF HAMi

China Merchants Bank boosts hardware pool utilization to 100% with Cloud Native Computing Foundation

China Merchants Bank, a leading commercial bank in China, faced significant challenges in efficiently managing its diverse and expanding AI workloads, which included model training, inference services, and R&D. Their heterogeneous compute platform, consisting of various AI accelerator cards, suffered from fragmented resource management, low hardware utilization, and wasted capacity on smaller tasks. To address this, they partnered with the Cloud Native Computing Foundation (CNCF) and utilized the HAMi project to build a more effective solution.

The solution implemented with CNCF's HAMi was a unified AI compute scheduling platform built on Kubernetes. This provided the bank with heterogeneous resource pooling, topology-aware scheduling to optimize performance, and fine-grained accelerator sharing. The measurable results included achieving 100% hardware pool utilization, a 30% reduction in cross-machine scheduling for distributed training, and the ability to partition accelerators into units as small as 1% of compute and 1 GB of memory, dramatically improving efficiency and scalability.

China Merchants Bank