Case Study: IBM Research democratizes access to LLMs with PyTorch's vLLM

IBM Research scales access to 1,300+ users with PyTorch-powered vLLM

The customer, IBM Research, faced the challenge of optimizing costly GPU utilization and providing its large research community with centralized, shared access to the latest experimental AI models. To meet this challenge, they built the RITS Platform using the vendor PyTorch's vLLM serving technology.

The solution implemented by PyTorch's vLLM provided high-performance inference serving with features like PagedAttention and Continuous Batching, which were critical for efficient resource management. The results for IBM Research were significant, with the platform growing to over 1300 active users hosting more than 100 models, all while achieving optimized GPU utilization and stable performance through a sophisticated autoscaling model driven by vLLM's metrics.

IBM Research