Case Study: eBay achieves scalable monitoring of petabytes of logs with Elastic Beats

Monitoring Petabytes of Logs per Day at eBay with Beats

eBay faced the challenge of monitoring an enormous and constantly changing environment, with 1.2 petabytes of logs per day and 5 million metric data points per second flowing from systems including Docker and Kubernetes. As its applications evolved, eBay needed a way to reliably collect, discover, and visualize logs and metrics across workloads, and turned to Elastic Beats, including Filebeat and Metricbeat, to help address the problem.

Elastic’s solution enabled eBay to ship logs and metrics from its containerized infrastructure, while eBay also built custom tooling like Collectbeat and an add_kubernetes_metadata processor to enrich data with Kubernetes context. The result was a more flexible monitoring pipeline for Sherlock.io, supporting growing data volumes with tiered retention and priority controls, even as organic logging growth reached 50% year over year.

eBay