Case Study: CERN CMS achieves rapid, scalable metadata search and aggregation with MongoDB

CERN CMS - Customer Case Study

CERN’s Compact Muon Solenoid (CMS) experiment—one of the Large Hadron Collider’s flagship detectors with thousands of physicists worldwide—generates immense, heterogeneous data (roughly 10 PB/year) spread across 100+ data centers and many relational and non-relational sources. Finding and combining the right data and metadata for analysis was slow and complex because information lived in varied formats and locations, and users often lacked the domain knowledge to locate it.

To solve this, CMS’s Data Management and Workflow team built the Data Aggregation System (DAS) on MongoDB, providing a schema-less, data-agnostic layer that accepts free-text/SQL-like queries, aggregates results from Oracle, PostgreSQL, CouchDB, MySQL and other sources, and caches aggregated responses. Running on a single 8-core server, DAS is used 24/7 worldwide, achieves about 6,000 documents/second for raw cache population, dramatically speeds information discovery, and is being prepared for horizontal scaling and broader deployment.

CERN CMS

Valentin Kuznetsov

Research Associate