Case Study: Diffbot achieves 70% cost savings and rapid, scalable real-time web-data processing with Amazon Web Services

Diffbot - Customer Case Study

Diffbot is a Palo Alto startup that provides APIs and tools to extract structured data from any web page for customers like The New York Times, Digg, and Salesforce. Its computer-vision and NLP-based analysis is CPU-intensive and subject to bursty, real-time traffic from news and social streams, and running and scaling on-premises infrastructure consumed too much time and capital for a fast-growing startup.

Diffbot moved core workloads to AWS, using compute-optimized EC2 instances (including Spot Instances), Amazon Route 53, AMIs stored on S3, and CloudWatch with Auto Scaling and predictive logic. The migration cut compute costs by about 70%, enabled five-minute scaling to handle hundreds of millions of pages per month, improved reliability and latency, and let the team refocus on developing machine-learning features.

Diffbot

Mike Tung

Founder and CEO