Case Study: Karlsruhe Institute of Technology (KIT) achieves scalable, unified management of heterogeneous nanoscopy metadata and provenance with ArangoDB

Karlsruhe Institute of Technology Managing Heterogeneous Metadata with ArangoDB

Karlsruhe Institute of Technology (KIT) needed a single, scalable system to manage the heterogeneous metadata produced by localisation microscopy (nanoscopy) experiments—both descriptive experiment metadata and provenance/workflow traces—stored and served from their Nanoscopy Open Reference Data Repository (NORDR). To address this, KIT adopted ArangoDB’s multi-model database (document, graph and key-value stores) to represent descriptive records, model provenance using the ProvONE standard, and harvest metadata across services.

Using ArangoDB, KIT persisted descriptive metadata in documents, captured workflow provenance as graphs, and maintained dataset associations in the key-value store, enabling uniform service design and easy sharding. The deployment runs about 90 GB of descriptive metadata across four ArangoDB shards, implemented the OAI‑PMH protocol on a NoSQL platform, produced peer‑reviewed provenance publications, and reports improved querying (AQL), graph traversal and scalable cluster operation—allowing KIT to scale its metadata framework further with ArangoDB.

Karlsruhe Institute of Technology

Ajinkya Prabhune

Karlsruhe Institute of Technology