Case Study: European Space Agency achieves the largest map of the Milky Way with InterSystems Caché

European Space Agency Is Creating the Largest Map of the Milky Way

The European Space Agency’s Gaia mission is creating the most detailed map of the Milky Way—over 30 billion observations and about 1.7 billion stars, growing to more than a petabyte of data by mission completion. That sheer volume plus the need to ingest daily telemetry (40–80 GB/day) and complete initial processing within 24 hours posed a major, time-sensitive data‑processing challenge for the ESA’s Science Operation Centre at ESAC in Spain.

InterSystems Caché was deployed as the working database to decompress, ingest and pre‑process incoming telemetry, verify instrument status, and perform initial image treatments and quality checks. Its high throughput and ability to access a roughly 40 TB working set enabled on‑the‑fly telemetry analysis and reliable delivery of quality‑checked data to the Gaia Data Processing and Analysis Consortium (DPAC), meeting the mission’s tight 24‑hour processing window and supporting large‑scale scientific analysis.

European Space Agency