Case Study: Global Consulting Firm Achieves 8x Faster Data Processing with Optiv

Optiv Empowers Global Consulting Firm to Expedite Data Processing Speed and Gain Customer Insight

Global Consulting Firm, a leading management consulting organization, needed a faster and more secure way to process proprietary customer data. Its on-premise ETL and analytics workflow was slow, highly manual, and siloed between data engineering and data science teams, while the firm also lacked the expertise to securely move the process into Azure Databricks. Optiv helped the firm modernize its data pipeline with its Secure Data Cloud service.

Optiv assessed the existing environment, built custom data engineering logic, standardized storage for processed data, and migrated and optimized the ETL process into Spark analytics on Databricks. The result was a unified, 100% cloud-based platform that increased ingest speed by 8x, cutting 400 GB of data processing from 16 hours to under 2 hours and 70 GB from 8 hours to 1 hour. Optiv also improved collaboration between data engineers and scientists, enabled scalable cluster sizing, and uncovered a major production bug during testing.