Case Study: GitLab achieves safer AI onboarding with Kipwise

GitLab strengthens AI onboarding testing in 2 weeks with Kipwise

GitLab, a company with a public and structured onboarding handbook, faced the challenge of ensuring its AI-powered company brain could be trusted by new hires for critical tasks. The risk was that a fluent but incorrect AI response could provide wrong policy information, skip approval steps, or leak sensitive data. To address this, they worked with vendor Kipwise to move beyond simple demos and implement a rigorous testing process before launch.

Kipwise implemented a solution centered on a defined acceptance test suite built from real onboarding tasks. This involved scoring the AI on five key behaviors: answer correctness, citation support, permission enforcement, refusal and escalation, and freshness. The result was a governed release process with non-negotiable gates for critical failures, ensuring the AI system was safe and reliable for new employees. This provided GitLab with more protection than a polished demo and created a repeatable framework for maintaining the system's accuracy over time.

GitLab