Case Study: Mohalla Tech Private Limited achieves 50% lower Spark costs and 2X faster OLAP jobs with DataPelago Accelerator for Spark

Mohalla Tech Private Limited speeds Spark pipelines 2x and cuts costs 50% with DataPelago

Mohalla Tech Private Limited, which powers the popular Indian social media platforms ShareChat and Moj, faced significant challenges with their Apache Spark-based data lakehouse. Their self-service "Cube Service," which processed terabytes of data daily for analytics, was plagued by performance bottlenecks and data skew. These issues caused compute-intensive jobs to fail and led to unpredictable costs and cascading failures. To address this, they turned to DataPelago and its DataPelago Accelerator for Spark product.

The solution, DataPelago Accelerator for Spark, was implemented rapidly without requiring code changes or data migration. It specifically optimized joins and aggregations over skewed data, which eliminated hotspots and stabilized their platform. As a result, previously failing jobs now completed successfully, job speeds increased by 2x, and costs were reduced by 50%, significantly surpassing their initial target. DataPelago enabled Mohalla Tech to fully unlock the potential of their open-source Spark infrastructure.

Mohalla Tech Private Limited