Case Study: Purdue University tracks 2.3M grocery products daily with Octoparse

Purdue University tracks 2.3M grocery products daily with Octoparse

Purdue University's Center for Food Demand Analysis and Sustainability (CFDAS) needed to collect daily data on over 2.3 million grocery products from 20 online grocery chains. This required a reliable and scalable web scraping solution to handle the immense volume of data, as manual collection or fragile custom scrapers were not viable options. They partnered with Octoparse to meet this challenge.

Octoparse implemented a solution that scrapes detailed product information daily from across 342 zip codes and five product categories. The scraped data is transferred in real-time to the university's data depot, enabling timely and efficient data management without manual intervention. As a result, CFDAS now provides agribusinesses, farmers, and policymakers with the real-time data and insights needed to make better decisions and improve the food system.

Purdue University