Case Study: Kaggle achieves a scalable, high-performance crowdsourcing platform with Microsoft Azure

Company Creates Crowdsourcing Platform in the Cloud to Solve Complex Problems

Kaggle is a crowdsourcing platform that connects organizations with data scientists to solve complex analytic problems. As the site grew, its original AWS/PHP/MySQL stack struggled with maintainability, rapid scaling and handling large data loads — problems that became urgent after landing a high‑profile $3 million competition and anticipating large spikes in users and traffic.

Kaggle migrated the platform to Microsoft Azure and rewrote the site in Visual C#, using Azure Compute, web and worker roles, blob storage and SQL Azure. The move — completed in about a month — made the site easier to update, highly scalable for sudden user and data surges, and more cost‑effective with pay‑as‑you‑go pricing; by early 2012 Kaggle had grown to over 32,000 users and attracted clients such as Allstate, NASA and Ford.

Kaggle

Anthony Goldbloom

Founder and Chief Executive Officer