Case Study: OpenEvidence achieves lower latency and faster deployments with Baseten

OpenEvidence delivers instant, accurate medical information with Baseten

OpenEvidence, a medical technology company, uses AI-powered search to deliver fast, accurate, and up-to-date medical information to clinicians at the point of care. As usage scaled to hundreds of thousands of doctors worldwide, the team needed help maintaining reliable performance during traffic spikes, reducing engineering time spent on model and infrastructure management, and accessing flexible compute without long-term cloud commitments. Baseten provided the inference infrastructure and model performance support they needed.

With Baseten’s Inference Stack, including Multi-cloud Capacity Management, Baseten Embeddings Inference, and training infrastructure, OpenEvidence was able to streamline deployment and improve model performance. The results were significant: 78% lower latency, 6x faster deployment processes, and an 8x+ reduction in infrastructure maintenance time, while also improving throughput and cost efficiency. Baseten now supports billions of requests per week for OpenEvidence, helping the team focus on its core product and mission.

OpenEvidence

Jagath Jai Kumar

Full Stack/ Engineer