Case Study: TomTom achieves faster, lower-cost AI inference with BentoML

TomTom cuts LLM latency and cost by 50% with BentoML

TomTom, a pioneering mapmaker and provider of location technology, faced the challenge of rapidly experimenting with and scaling its generative AI initiatives while maintaining focus on its core navigation services. They partnered with BentoML and its AI application framework to streamline model serving and deployment, seeking more control, cost-effectiveness, and efficiency beyond initial prototypes.

Using BentoML's framework, particularly its OpenLLM and BentoCloud platform, TomTom integrated various models into a cohesive service, which simplified model composition and inference. The solution delivered significant performance improvements, including an average 50% reduction in both latency and cost while maintaining quality. BentoML enabled TomTom to accelerate experimentation, swiftly test new AI services, and scale them rapidly.

TomTom