Case Study: Box defines launch readiness and ships AI with Braintrust

Box defines launch readiness with Braintrust using 80%+ launch gates and a 330-question golden dataset

Box, an intelligent content management platform, faced the challenge of manually evaluating its new AI product, the Box Agent, which was an inefficient and unscalable process. They needed a robust system to measure quality and define clear launch readiness criteria to ship with confidence. They turned to Braintrust to implement an evaluation platform.

Using Braintrust, Box built a programmatic evaluation practice with versioned datasets and custom graders. This solution enabled them to run nightly evals, identify regressions, and establish an 80% score threshold on a 330-question golden dataset for launch. Braintrust provided the system of record that allowed Box to successfully analyze failures, compare models, and ultimately launch their AI agent with a deterministic measure of quality.

Box