Case Study: Slack achieves real-time security visibility with StarTree and Apache Pinot

Slack cuts data latency from 24–48 hours to under 1 second with StarTree

Slack identified a challenge in providing its enterprise customers with real-time visibility into data security. The existing batch-processing system, powered by Apache Pinot and provided by StarTree, had a 24- to 48-hour data delay, which was unacceptable for monitoring potential data exfiltration by external apps as it happened.

StarTree's solution involved building a new real-time ingestion pipeline using Kafka and Apache Pinot. This upgrade allowed Slack to process 400-500 million records in real time and perform complex aggregations. The results were significant, with data latency reduced to under one second and query latency under 10 seconds, giving customers immediate security visibility.

Slack