Octoparse Customer Stories

In Partnership with

Purdue University

Agriculture

United States

Octoparse understands the need of the center and stays close to develop the program which put the needs into practice.

— Jinho Jung

Research Associate of The Center for Food Demand Analysis and Sustainability and Department of Agricultural Economics, Purdue University

The Center for Food Demand Analysis and Sustainability (CFDAS) is a research center of the College of Agriculture, Purdue University. The center facilitates the flow of data about consumers and food markets so consumers can make more informed food choices. Meanwhile, agribusinesses, policymakers, and farmers can improve the food system.

The main challenge the center faced was how to grab information about over 2 million products from 20 online grocery chains on a daily basis. It necessitates faster servers, larger data storage, and real-time data transfer. In addition, it needs to aggregate all the data to its interactive dashboard then its audiences can monitor the prices of grocery items daily and across regions.

The center is now applying Octoparse to its daily web scraping task to pull a considerable amount of data from 20 online grocery chains for five categories of grocery items around 342 Zipcodes. The center's data depot is also connected to Octoparse's data storage to get real-time data without delay.

“The center needs to collect data from 20 online grocery chains... for 5 categories of grocery items, around 342 Zipcodes every day. It aggregates up to 2.3 million products daily.”

“Octoparse developed a scraping program for detailed grocery information such as items, categories, and geographies...(With the scraped data) the center and the dashboard will help producers, agribusinesses, and policymakers make decisions that will improve the food system, as well as guide research on nutrition and plant innovations.”

“Octoparse links its data storage with the center’s data depot for transferring the scraped data every day so that the center can manage the data in a timelier manner.”

“Octoparse did a great job not only on data scrapping but also on understanding projects and the center's needs. Data collection scrapped from online spaces would have been useless unless Octoparse understands the whole purpose of the project and the needs of their client.”

—— Jinho Jung, 

Research Associate of the Center for Food Demand Analysis and Sustainability

and Department of Agricultural Economics, Purdue University

Now the center is providing agribusinesses, farmers, policymakers, etc., with timely, accessible data and insights across food prices, food production and supply, consumer food spending, and consumer preferences, attitudes, knowledge, and beliefs. By leveraging data, Octoparse helps organizations like this create value and make the most of their resources.

Price tracking & comparison in marketing consultancy

Inventory tracking for retail business

Ecommerce product tracking for successful reselling

We work closely with you to understand your data requirements and make sure we deliver exactly what you need. 

Copyright © 2023 Octopus Data Inc. All Rights Reserved.