Case Study: Sully.ai cuts inference costs and latency with Baseten

Sully.ai cuts inference costs 90% with Baseten

Sully.ai, a healthcare technology company, faced significant challenges with latency, unsustainable inference costs, and inconsistent model quality when using closed-source models to power their suite of AI clinical agents. These limitations were critical for their real-time, clinical-grade use cases and threatened their ability to scale efficiently.

To address this, Sully.ai transitioned its inference stack to open-source models running on the Baseten platform with NVIDIA Blackwell. This move provided Baseten's rapid deployment of newly released models and deep performance optimization expertise. The results were substantial: a 90% reduction in inference costs, a 65% decrease in median latency, and the return of over 30 million clinical minutes to their customers' workforce, demonstrating a 21x return on agent spend.

Sully.ai