Case Study: The Able Few achieves scalable, cost-effective real-time processing of billions of social media messages with DigitalOcean

How The Able Few Uses DigitalOcean

The Able Few, working for client Fizziology, faced a massive data challenge: store and serve 15 billion documents with growth of about 2 million documents per day (roughly 6 TB) and sub‑half‑second retrieval, while absorbing huge traffic spikes from social feeds. Their prior provider (Rackspace) was too costly and its HDDs couldn’t keep up, so they needed a more affordable, high‑I/O, instantly scalable hosting solution.

They moved to DigitalOcean SSD droplets and built “Centrifuge,” a parallel stream‑processing pipeline (Storm, Elasticsearch, Redis) with data replicas for failover and quick scaling via Images/API. A five‑node test outperformed a twelve‑node Rackspace cluster by 23×, and production now runs ~42 nodes, maintains a 24‑month rolling index, processes ~2.8M tweets/day (1.5B+ items analyzed), and achieved much higher throughput at about one‑third the cost.

The Able Few

Iouri Kostine

Partner