Web scraping price collection and Detailed Average Prices - European Statistical System (ESS)

​Eurostat and Statistics Netherlands hosted a conference on "Internet price collection and Detailed Average Prices" in The Hague on 20 and 21 June 2013 to discuss the latest developments on Internet price collection and Detailed Average Prices. The conference gathered representatives from a large number of National Statistical Institutes and the Commission (Eurostat and DG SANCO).

 
The aim of the conference was to discuss on-going research on methods of automatic data collection on consumer prices from the Internet, in the context of a multi-purpose approach to price statistics. The approach reuses prices collected for one statistic, e.g. HICP, in the production of other statistics, e.g. PPP and Detailed Average Prices. This can only be achieved when basic information on prices and products are structured and harmonised in both temporal and spatial dimensions which is the main challenge in the Detailed Average Prices project.

Statistics Netherlands, DESTATIS and ISTAT presented their work on the develment of web scraping techniques (computer software techniques for extracting information from websites) for price collection. In the conference, they shared their experiences so far and gave great insight into current and future challenges in this area.

 
The aim of the Detailed Average Prices project is to produce average consumer price levels for a detailed and well defined set of products. The results from pilot projects since 2009 have been published by Eurostat, and by DG SANCO in their "Consumer Markets Scoreboard". DG SANCO gave insight into the use of Detailed Average Prices in their monitoring of product and sector markets. Representatives from several National Statistical Institutes explained and elaborated challenges faced by this task.

 
Presentations related to the conference are available in CIRCABC in the public domain WSMPPS Meeting 2013-06 The Hague.

For further information: