Archive:EU statistics on income and living conditions (EU-SILC) methodology – sampling

This article is part of the Eurostat online publication EU statistics on income and living conditions (EU-SILC) methodology, which offers a description of the dimensions along with the EU-SILC indicators are disseminated in Eurostat's dissemination database within the overall domain of Income and living conditions. Since the indicators are of multidimensional structure are analysed simultaneously along several dimensions, and published in separate datasets along with different combinations of dimensions. The articles presents these dimensions and provides information on their methodological limitations.

Table 1: Sampling frames new
Source: National Intermediate Quality Reports 2011

Table 2: Main characteristics of countries’ sampling designs
Source: Eurostat

Table 3: AROPE indicator (2013) standard error and 95% confidence intervals
Source: Eurostat

Table 4: Estimated standard errors for estimators of net change in the AROPE between 2008 and 2013
Source: Eurostat

Introduction

EU SILC is a sample survey. The legislation specifies that data shall be based on nationally representative probability samples and prescribes minimum effective sample sizes, but leaves to the country the choice of a specific sampling design. This articles describes the main characteristics of the sampling design and the sampling error for the main EU-SILC indicator: AROPE.

Sampling frames

The big strength of EU-SILC is the usage of the best sampling frames available in each National Statistical Institute (NSI). According to the framework, data are to be based on a nationally representative probability sample of the population residing in private households within the country, irrespective of language, nationality or legal residence status. All private households and all persons aged 16 and over within the household are eligible for the operation. Persons living in collective households and in institutions are generally excluded from the target population. The sampling frame as well as methods of sample selection should ensure that every individual and household in the target population is assigned a known probability of selection that is not zero. As shown in Table 1, the vast majority of countries used for the 2011 EU-SILC operation population registers, or national census or a master sample derived from the census.

Sampling design

The sample design describes all the steps to be carried out when selecting a sample of households or persons. It aims to improve the quality of the estimates produced and to control costs. Various strategies are in place in different countries to achieve this objective. The table 2 summarizes the sampling design used in each country for the 2011 operation. Countries choose a specific sampling design according to the structure of the country and the population, according to existing information and taking into account budgetary constraints. The most used sampling design is stratified multistage sampling. Only five countries do not use stratification criteria to draw their sample. In details: Malta, Denmark and Island use a simple random sample design and Sweden as well as Norway use a systematic sample. Concerning all the remaining countries, they apply one or more stratification criteria, mainly a geographical stratification. Among them, the majority uses a multi-stage sampling with the exception of Luxembourg, Germany, Cyprus, Slovakia, Switzerland, Austria and Lithuania which use a stratified simple random sample. Estonia uses a systematic stratified sample and Hungary is the only country to apply a different sampling design for drawing each rotational group. Countries send every year to Eurostat general information on the sampling design used and detailed information at the level of microdata on the strata and PSU from which each household is drawn. The efficiency of the sampling design has a big impact on standard error and should be monitored over time. On the other side, changing it is extremely costly.

Standard error estimation

Given the high policy relevance of EU-SILC there is increasing demand from the stakeholders for accuracy measures of the published indicators and for measures of the significance of net change of indicators over time for correct monitoring of the evolution of social exclusion phenomena. As seen, EU-SILC is a complex survey involving different sampling design in different countries. For this reason, "to the book" standard methods for calculating accuracy measures are not directly applicable. Eurostat with the substantial contribution of Net-SILC2 has put in place a simple method for standard error estimation based on linearization and coupled with the ultimate cluster approach. For further details on the method, please consult the working paper Standard error estimation for the EU–SILC indicators

Results

We have applied the method for estimating the standard error and confidence intervals on the indicator AROPE (At-risk-of poverty or social exclusion). This indicator is the proportion of persons being in one or more of the three following situations: at-risk-of poverty, i.e. below the national poverty threshold (60% of median national equivalized income), severely materially deprived, living in a household with very low work intensity. We have considered this indicator as a proportion making the assumption that the poverty threshold is a fixed amount and equal to the point estimate. According to the characteristics and availability of data for different countries we have used different variables to specify strata and cluster information. In particular, countries have been split into three groups:

1) Belgium, Bulgaria, Czech Republic, Ireland, Greece, Spain, France, Croatia, Italy, Latvia, Hungary, the Netherlands, Poland, Portugal, Romania, Slovenia and the United Kingdom whose sampling design could be assimilated to a two stage stratified type we used DB050 (primary strata) for strata specification and DB060 (Primary Sampling Unit) for cluster specification;

2) Germany, Estonia, Cyprus, Lithuania, Luxembourg, Austria, Slovakia, Finland, Switzerland whose sampling design could be assimilated to a one stage stratified type we used DB050 for strata specification and DB030 (household ID) for cluster specification;

3) Denmark, Malta, Sweden, Iceland, Norway, whose sampling design could be assimilated to a simple random sampling, we used DB030 for cluster specification and no strata.

The approach used can take account of stratification, multi-stage selection, unequal probabilities of inclusion for the sample units and re-weighting for unit non-response. However it does not reflect the gain in accuracy caused by calibration weighting. The effect of calibration on variance could be significant especially in the countries where powerful auxiliary information from income registers has been used to adjust the sampling weights. This in some cases may lead to overestimation of sampling errors. Results are shown in Table 3.

The same approach has been used to calculate variance of net change over two different years. In order to monitor the process towards agreed policy goals, particularly in the context of the Europe 2020 strategy, users are particularly interested in the evolution of social indicators. However, interpreting differences between point estimates at different wave may be misleading. It is therefore necessary to estimate the standard error for these differences in order to judge whether or not the observed differences are statistically significant.

Estimated standard errors and confidence intervals (based on normality assumption) for net changes in the AROPE between 2008 and 2013 are shown in Table 4. If a confidence interval does not include 0, we can say the difference in the AROPE between 2008 and 2013 is statistically significant (at a given level of confidence).

Further Eurostat information

Publications

Main tables

Income and living conditions (t_ilc)

Database

Living conditions and welfare (livcon), see:

Income and living conditions (ilc)

People at risk of poverty or social exclusion (Europe 2020 strategy) (ilc_pe)

Main indicator - Europe 2020 target on poverty and social exclusion (ilc_peps)

Dedicated section