Case Study: Esri achieves real-time, high-throughput geospatial analysis with Elastic

How Elasticsearch is SPARKing Our Geospatial Analysis

Esri, a leader in GIS software, needed to process high-velocity spatiotemporal observation data from moving objects and sensor networks in real time. The challenge was to sustain tens of thousands of events per second per node, scale throughput linearly as nodes were added, handle bursty data, and support complex queries across id, time, space, and attributes while enabling visualization and batch analytics.

Esri implemented Kafka + Spark Streaming for ingestion and streaming analytics, Elasticsearch with the Spark Elasticsearch Connector for scalable geospatial storage and search, GIS Tools for Hadoop for spatial operations, and the ArcGIS API for JavaScript for visualization. Benchmarks showed 132k events/s on one ingestion node (282k on two nodes) and up to 249k writes/s to a five-node Elasticsearch cluster; the solution delivers continuous spatial aggregations, fast geolocation queries, and batch analytics for real-world use cases like Port of Rotterdam vessel, density, and dredging analyses.

Esri

Adam Mollenkompf

Esri