Case Study: Goldman Sachs achieves real-time trade flow visibility and scalable monitoring with Elastic

How the Elastic Stack Changed Goldman Sachs

Goldman Sachs began using Elasticsearch organically across its engineering teams and formed a centralized Elasticsearch engineering group as usage grew. The firm faced a business challenge to track trade flow like a manufacturing pipeline across a complex, distributed system: teams needed real‑time visibility into where messages were, timeliness metrics, alerts for delays or dropped messages, and analytics to pinpoint inefficiencies and support what‑if analysis.

GS implemented a Center of Excellence that standardized and hardened open‑source Elasticsearch with GS plugins, provided centralized provisioning and self‑service APIs, and integrated monitoring, governance and vendor support. For trade tracking they instrumented clients to emit JSON trackers into LARA → Kafka, sharded flows into Elasticsearch with Redis queues and Esper for real‑time detection, and exposed Kibana dashboards and alerters. The solution scales to tens of flows and billions of documents (≈6B docs, 4 TB primary across 22 nodes, ~45M docs/day), delivering real‑time search, alerts and analytics to pinpoint hotspots, measure timeliness and drive continuous improvement.

Goldman Sachs

Haiying Guo

Vice President in Engineering