Case Study: DDI achieves faster, more accurate behavioral simulation analysis with Databricks

Helping companies develop exceptional leaders

DDI, a global leader in leadership development and assessment, needed to automate the scoring of behavioral simulations that had long been evaluated by human assessors. Working with Databricks and its Data Intelligence Platform, DDI set out to use machine learning and GenAI to speed up evaluation and improve the accuracy of behavioral simulation reports.

Databricks helped DDI manage the full ML lifecycle with Databricks Notebooks, MLflow, Unity Catalog, and Model Serving to experiment, track, govern, and deploy models at scale. The results were strong: prompt optimization with DSPy lifted recall from 0.43 to 0.98, and an instruction-tuned Llama3-8b model reached an F1 score of 0.86 versus a 0.76 baseline. Databricks also enabled secure, scalable deployment with auto-scaling and serverless capabilities, helping DDI improve reliability and streamline operations.

DDI

Chris Coughlin

Senior Manager of Assessment Content Design and Development