Case Study: University of Bristol benchmarks LLM emotional intelligence with Prolific

University of Bristol benchmarks LLM emotional intelligence with Prolific and achieves 88% human-AI agreement

The University of Bristol researchers faced the challenge of benchmarking the emotional intelligence of large language models (LLMs) against human judgment. Their task involved managing a complex, multi-phase study with significant ethical concerns around exposing participants to sensitive content and required sophisticated technical infrastructure to handle real-time data collection. They turned to Prolific to source high-quality human data for this benchmark.

Prolific provided the solution with its precision participant targeting, participant-group management for phased studies, integrated ethical content warnings, and throttled-access capability to manage server load. Using Prolific, the researchers efficiently recruited 1,000 participants and found a remarkable 88% correlation between human and AI rankings of emotional events. This high-quality data demonstrated Prolific's value in enabling trustworthy and efficient AI benchmarking research.

University of Bristol