Case Study: China Merchants Bank boosts AI efficiency with CNCF cloud native stack

China Merchants Bank boosts accelerator utilization to 60%+ with CNCF

China Merchants Bank, a leading commercial bank in China, faced a significant challenge in efficiently managing its AI platform across nearly 10,000 heterogeneous accelerator cards. The bank needed to support large-model training, fine-tuning, and online inference workloads more effectively, as simply adding more hardware would not solve the underlying issues of low utilization, operational isolation, and the high cost of duplicating resources for multi-tenant fine-tuning.

With the Cloud Native Computing Foundation's technologies, including Kubernetes, Kueue, KEDA, HAMi, and Fluid, the bank built a unified control plane with decoupled execution paths. This composable cloud native stack from the Cloud Native Computing Foundation allowed for distinct workload policies, leading to a more than 60% increase in average accelerator utilization. The solution also reduced inference cost per 1M tokens by over 60% and increased multi-tenant training density fivefold by enabling five tenants to share a single base-model instance.

China Merchants Bank