Case Study: Cursor achieves real-time low-latency inference with Together AI

Cursor launches real-time inference at scale in days with Together AI

Cursor, an AI-native developer tools company, faced the challenge of delivering real-time, low-latency inference for its AI-powered coding platform. In-editor agents needed to respond within the developer's feedback loop, making worst-case latency under concurrency a hard requirement. To meet this, Cursor partnered with Together AI to leverage their GPU Clusters infrastructure.

Together AI implemented a production inference solution on the new NVIDIA Blackwell architecture (GB200 NVL72). They optimized the entire stack, including ARM host software and custom kernels, and established a rapid quantization pipeline using TensorRT-LLM and NVFP4 to preserve model quality while reducing latency. This collaboration enabled Cursor to deploy production workloads across multiple data centers and establish a repeatable process to move from new model weights to a test endpoint in just days.

Cursor