Case Study: ByteDance achieves 100K+ QPS agent memory and 7-day-to-1-day training with LanceDB

ByteDance cuts AI pipeline time from 7 days to 1 day with LanceDB

ByteDance's Volcano Engine, its cloud and AI service platform, faced significant bottlenecks in its AI data infrastructure. Its existing data lake, built for analytics, struggled with the demands of AI workloads, including frequent schema changes, multimodal data types, and inefficient vector index training that required massive single-node servers and took days. Separately, its team building the ArkClaw agent framework needed a scalable, production-grade memory backend. They turned to vendor LanceDB and its core Lance format to rebuild their stack.

The solution involved rebuilding Volcano Engine's entire AI data platform on LanceDB, using the Lance format as its core storage. This provided native multimodal support and enabled distributed index training, which eliminated single-node bottlenecks. For the ArkClaw agent memory, they built a high-performance layer on LanceDB capable of hybrid search and versioned memory using a feature called GitforMemory. The results were substantial: a model training pipeline was reduced from 7 days to 1 day, GPU utilization rose to 95%, and the LanceDB-powered agent memory layer handles over 100,000 queries per second in production.

ByteDance