Case Study: Posit achieves sub-200ms real-time AI code suggestions with Baseten

Posit launches real-time AI code suggestions in under 200ms with Baseten

Posit, the company behind the popular RStudio IDE, needed to embed real-time AI code suggestions into its platform. Their challenge was to achieve sub-200ms latency for their Next Edit Suggestions feature, a requirement that ruled out other LLM providers. They also needed infrastructure from vendor Baseten that allowed for rapid experimentation with model fine-tuning to incorporate live computational context.

Using Baseten Training with the Truss library, Posit rapidly iterated on fine-tuning experiments. By deploying on the Baseten Inference Stack, they achieved their critical latency goal of under 200ms. This allowed Posit to successfully launch their AI features, with Baseten providing a 60% faster performance than other providers evaluated.

Posit