Archive:Sample size and non-response - quarterly statistics
Data extracted in September 2020
Planned article update: December 2020.
Highlights
During the first quarter of 2020, as a consequence of the COVID-19 outbreak, the data collection (i.e. interviews) for the European labour force survey (EU-LFS) have been severely hampered in most countries. This resulted in a sharp increase of the non-response, especially in the last month of the first quarter 2020 (i.e. March 2020), and consequently in a downsizing of the achieved sample. The reduced participation in the survey also led to an increase of the sampling errors, that adversely affect the precision of the estimates.
In this context, this article assesses the quality of the data gathered through the EU-LFS for the European Union (EU) as a whole, for each EU Member State individually, as well as for the United Kingdom, two EFTA countries (Norway and Switzerland) and four candidate countries (Montenegro, North Macedonia, Serbia and Turkey). Analysis is based on data and information available up to 10 September 2020. Data of the first quarter of 2020 is mainly compared to data of the first quarter of 2019 (for sample size and sampling errors) in order to avoid the bias due to seasonality. Comparison with annual averages of previous years is also included regarding the non-response. Please note that this comparison is reliable as the non-response is not particularly affected by seasonality and is usually quite stable throughout the year.
This article is part of the online publication Labour market in the light of the COVID-19 pandemic - quarterly statistics alongside namely with the articles Labour market slack - unmet need for employment and Hours of work.
Full article
Sharp increase in the unit non-response rate in the first quarter of 2020
The unit non-response occurs when no data are collected about a population unit (usually a person or a household) designated for data collection. Consequently, the non-response rate is the ratio of the number of units for which data have not been collected to the total number of units designated for data collection.
Since the beginning of the time series (2011), the non-response rate smoothly increased at EU level (see Figure 1). The same conclusion can be drawn when considering all countries carrying out the European Labour Force Survey (EU-LFS). However, between the year 2019 and the first quarter of 2020, the non-response rate sharply increased at EU level: from 24.7 % in 2019 to 31.9 % in Q1 2020 (+7.2 percentage points). This can be explained by the lock down measures adopted by most Member States during the first quarter of 2020 to cope with the COVID-19 pandemic. Face-to-face (CAPI and PAPI) data collection methods have been stopped and replaced as much as possible by remote collection methods (CATI or CAWI). The unit non-response mainly raised due to phone numbers or email addresses which were not always immediately available. The rise is a bit softer when all EU-LFS participating countries are considered: from 27.8 % in 2019 to 32.2 % in Q1 2020 (+4.4 percentage points).
Even if a sharp increase between the year 2019 and the first quarter of 2020 can be seen at EU level, the situation is quite difference across countries. In Bulgaria and Ireland, the unit non-response rate increased by around 10 percentage points (12 p.p. and 9 p.p. respectively). Belgium, Czechia, Greece, France, Latvia, Hungary, Malta, Portugal, Slovenia, the United Kingdom, North Macedonia, Serbia and Turkey also recorded a significant rise in their unit non-response rate (rise between 3.5 p.p. and 7.2 p.p.) (see Figure 2). All these countries broadly used face-to-face interviewing techniques (CAPI and PAPI) for the EU-LFS data collection. By contrast, countries relying exclusively on CATI and CAWI techniques did not register a big impact on their response rate between the year 2019 and the first quarter of 2020. Denmark and Finland saw their unit non-response rate increased by 1.0 p.p. and 1.8 p.p. respectively. In Luxembourg (-4.3 p.p.), Sweden (-0.2 p.p.), Norway (-0.3 p.p.) and Switzerland (-1.0 p.p.), the unit non-response rate even decreased.
<sesection>
Sample size decreased starting from the 10th week of 2020
The EU-LFS is based on the uniform sample distribution of the quarterly sample over all the reference weeks of the quarter. This means that the size of each weekly sample corresponds to the size of the total quarterly sample divided by the number of the weeks of the quarter (generally 13).
The distribution of the achieved sample size in Q1 2020 is similar to Q1 2019 for the first 9 weeks of the year. The weekly difference in the collected sample between the two years is on average around 4 000 individuals for a total situated between 75 000 and 85 000 individuals (see Figure 3). From week 10, the difference between the two years increased to 12 000 in week 10, and even to 14 000 in weeks 11 and 12. In week 13, the difference between the two years narrowed to around 7 000 individuals.
These numbers reflect the lock-down measures taken by the governments of several EU Member States in the first quarter of 2020 from week 10 onwards. The achieved sample size shows a decrease from week 10 due to restrictions in surveys data collection. Then, after countries adopted measures to increase the survey participation (like the use of additional information to reach people selected in the sample: as telephone numbers or email addresses from registers of mobile phone or landlines numbers and tax databases), the achieved sample size increased again.
In the first quarter of 2020, many countries reported a reduction of the sample size collected in weeks 10-13 compared to weeks 1-9 (see Figure 4). Only 6 EU Member States (Finland, Malta, the Netherlands, Estonia, Croatia and Luxembourg) had a bigger average weekly sample in weeks 10-13 than in weeks 1-9.
At EU level, the ratio of the average weekly number of people interviewed in the weeks 10-13 to the average weekly number of people interviewed in the weeks 1-9 in 2020 was 89.8%, while it was 99.1% in 2019. This corresponds to a decrease of nearly 10 percentage points.
The situation was particularly critical in Slovenia, where the ratio of the average weekly sample size in the weeks 10-13 to the weeks 1-9 was 56% in 2020, being 99% in 2019. Bulgaria also recorded a ratio less than 60% in 2020 (55%) but the difference with its ratio in 2019 (75%) is smaller. The decrease is the average weekly sample size is also noticeable in Ireland and Latvia, with a ratio of average sample size in the weeks 10-13 to the weeks 1-9 less than 80% in 2020 (74% in Ireland and 78% in Latvia) and close to 100% in 2019 (98% in Ireland and 97% in Latvia).
Slight increase in sampling errors in Q1 2020 figures
The estimates produced by the EU-LFS are subjected to specific precision requirements specified in the Council Regulation 577/1998, which establishes the organisation of the survey. In particular, specific requirements concern the reliability of estimates on the number of employed and unemployed. The Coefficient of Variation (CV) for the employed and unemployed population, as indicator of the precision of the EU-LFS estimates, is transmitted by countries to Eurostat on a quarterly basis.
As can be seen from figure 5, the COVID-19 crisis also had an impact on precision of the measurement of employment and unemployment. Almost all countries show a deterioration of the precision of the estimates for both employment and unemployment between the first quarter of 2019 and the first quarter of 2020. This deterioration is linked to the continuous increase of the unit non-response (already visible in previous years), which reduces the sample size, especially in the specific segment of the population attached to the labour market, but also to an additional increase of bigger intensity coming from the COVID outbreak.
The loss of precision in Q1 2020, compared with Q1 2019, mostly affects Bulgaria, Greece, Spain, Italy, Romania and Sweden for both estimates on employment and unemployment. Among the non-EU countries, Norway and Switzerland also recorded an important loss of precision for both estimates in Q1 2020 compared with Q1 2019. In addition, Croatia, Cyprus and the Netherlands are affected by a loss of precision for the unemployment estimates.
Source data for tables and graphs
Data sources
All figures in this article are based on quarterly results from European labour force survey (EU-LFS).
Source: The European Union labour force survey (EU-LFS) is the largest European household sample survey providing quarterly and annual results on labour participation of people aged 15 and over as well as on persons outside the labour force. It covers residents in private households. Conscripts in military or community service are not included in the results. The EU-LFS is based on the same target populations and uses the same definitions in all countries, which means that the results are comparable between countries.
European aggregates: EU refers to the sum of EU-27 Member States. If data are unavailable for a country, the calculation of the corresponding aggregates takes into account the data for the same country for the most recent period available. Such cases are indicated.
Definitions: The concepts and definitions used in the labour force survey follow the guidelines of the International Labour Organisation.
Five different articles on detailed technical and methodological information are linked from the overview page of the online publication EU labour force survey.
Context
The COVID-19 virus hit Europe in January and February 2020, with the first cases confirmed in Spain, France and Italy. COVID-19 infections have now been diagnosed in all European Union (EU) Member States. To fight the pandemic, EU Member States have taken a wide variety of measures. From the second week of March, most countries closed retail shops apart from supermarkets, pharmacies and banks. Bars, restaurants and hotels have also been closed. In Italy and Spain, non-essential production was stopped and several countries imposed regional or even national lock-down measures which further stifled the economic activities in many areas. In addition, schools were closed, public events were cancelled and private gatherings (with numbers of persons varying from 2 to 50) were banned in most Member States. The large majority of the prevention measures were taken during mid-March 2020 and most of the prevention measures and restrictions were kept for the whole of April and May 2020. The first quarter 2020 is consequently the first quarter in which the labour market across the EU has been affected by COVID-19 measures taken by the Member States. Employment and unemployment as defined by the ILO concept are, in this particular situation, not sufficient to describe the developments taking place in the labour market. In this first phase of the crisis, active measures to contain employment losses led to absences from work rather than dismissals, and individuals could not search for work or were not available due to the containment measures, thus not counting as unemployed.
Direct access to