Case Study: Databricks achieves faster, lower-cost model evaluation with SuperAnnotate

Databricks cuts evaluation costs 10x with SuperAnnotate

Databricks, a leading data and AI company, faced challenges in evaluating its retrieval augmented generation (RAG) systems due to the subjective nature of judging LLM responses and the difficulty of collecting large-scale, clean feedback data. To build a reliable and cost-effective "LLM as a judge" system, the company partnered with SuperAnnotate to overcome these obstacles.

SuperAnnotate provided a highly customizable data annotation platform to create a standardized grading rubric and purpose-built tooling for RAG evaluation. This collaboration enabled Databricks to efficiently collect and process feedback data, refining their model prompts for consistency. As a result, SuperAnnotate's solution allowed Databricks to use a more cost-effective model for evaluations, leading to a tenfold reduction in costs and a threefold increase in speed.

Databricks