Case Study: Zoom achieves 82% infrastructure cost reduction and faster GDPR deletes with Onehouse

Scaling the Data Lakehouse to 100TB/day while Meeting GDPR Requirements

Zoom, the largest video calling platform in the US and UK, faced significant challenges scaling its data infrastructure to handle over 100TB of daily log data while meeting strict GDPR requirements. Its previous architecture was time-consuming for both data ingestion and for servicing user data deletion requests, which could take multiple hours to complete. This created operational inefficiency and compliance risks for the company.

To solve this, Zoom worked with Onehouse to implement a solution using Apache Hudi on Amazon S3. This new data lakehouse architecture, powered by Spark Structured Streaming jobs, optimized data ingestion and enabled highly efficient data deletion through Hudi's Bloom filter indexes. The results were transformative: Zoom achieved an 82% reduction in infrastructure costs, accelerated data ingestion to 150 million messages every five minutes, and slashed the time to delete 1,000 records from 3 hours down to just 1-2 minutes.

Zoom Video Communications