Case Study: IBM improves Gen AI model performance with Sama

IBM improves Gen AI model validation with Sama, achieving a 95% acceptance rate

The customer, a Fortune 500 tech leader known for popular content-sharing mobile apps, sought to improve the performance of its generative AI foundation models for the North American market. It faced challenges with caption writing and prompt-and-response evaluation, lacking the necessary expertise in contextual nuance and English language proficiency to achieve high model accuracy. It partnered with vendor Sama for their gen AI evaluation services.

Sama implemented a solution combining partial automation with a human-in-the-loop approach. Their team evaluated and refined image captions and LLM prompts/responses for errors, logic, tone, and regional context, creating high-quality data to fine-tune the models. This led to a 95% client acceptance rate, significantly fewer hallucinations, improved grammar and context, and faster model validation, ultimately enhancing the end-user experience.

IBM