Case Study: IBM achieves real-time, self-service DataOps and global network visibility with IBM StreamSets

How Self-service Data Supports Operational Excellence

IBM manages a massive global network spanning 1,000+ sites and terabytes of telemetry per day, with some devices producing 10–12K records/sec. To replace a code-heavy Logstash setup and enable DataOps and self‑service data at the edge, IBM evaluated and adopted IBM StreamSets (the StreamSets DataOps platform) to reliably ingest, transform and route high‑volume, heterogeneous network data to regional data lakes for real‑time use.

IBM StreamSets was rolled out to build and run pipelines on StreamSets pods with automated provisioning agents, pipeline template fragments/topologies, and CI/CD integration with Docker and GitHub. The deployment delivered measurable impact: HQ now has visibility into 20,000+ pipelines and billions of streaming records, local teams get immediate access to the data they need, and IBM reports that IBM StreamSets handles their volume at scale while enabling self‑service and faster operations.

IBM

Divya Yashwanth

Software Developer, CIO Network Engineering