Case Study: Graphite builds reliable AI code review at scale with Braintrust

Graphite reduces negative rules by 5% with Braintrust

Graphite, a company that provides developer tools, faced the challenge of ensuring their AI-powered code reviewer, Diamond, delivered consistently precise and actionable feedback without hallucinations. To build a reliable evaluation framework and move beyond ad-hoc manual processes, they partnered with Braintrust.

Using Braintrust, Graphite implemented a systematic evaluation solution. They curated datasets from real developer interactions and built custom scoring functions. This enabled them to run rapid experiments and make data-driven deployment decisions. The results included a 5% reduction in negative rules generated and the achievement of a target 90%+ acceptance rate, allowing them to ship reliable AI features quickly.

Graphite