ESS Reference Metadata Reporting Standards

You will find here a general introduction to the standards used within the European Statistical System (ESS) to describe the data sets disseminated on Eurostat's website, both in terms of content and quality. This specific type of information is called reference metadata.

The first sections below explain the differences between structural and reference metadata and describe the various metadata reporting structures within the ESS. Then, the IT-platform developed by Eurostat for storing and disseminating ESS metadata is presented.

Structural versus reference metadata

Two types of statistical metadata can be distinguished: structural metadata and reference metadata.

Structural metadata are used to identify statistical data, e.g. titles, subtitles, short descriptions, dimension names, variable names, dimensions, etc.

For example, structural metadata include the titles of the variables and dimensions of statistical datasets, as well as the units of measurement used (e.g. EUR), code lists (e.g. for territorial coding), data formats, potential value ranges, time dimensions, value ranges of flags, classifications used, etc.

Data and structural metadata MUST go together. A number such as 3,651,881 is totally meaningless unless you are provided with its accompanying structural metadata, namely that it concerns the total female population of Bulgaria on the 1st January 2017

Reference metadata describe statistical concepts and methodologies used for the collection and generation of data, and provide information on data quality.

They thus assist users with the interpretation of the data, thereby being strongly content-oriented.

Contrary to structural metadata, reference metadata can be decoupled from the data, i.e. they can be generated, collected or disseminated separately from the statistics to which they refer.

They can also be associated with different levels of data: entire collections, data sets from a given country, or for a data item concerning one country and one year.

Illustration

Main reference metadata reporting standards

The main reference metadata reporting standards used within the ESS are the following:

SIMS (Single Integrated Metadata Structure)

SIMS is the dynamic inventory and conceptual framework for all ESS quality and reference metadata concepts.

In this structure, all statistical concepts of the two existing ESS reporting structures (ESMS and ESQRS - see below) have been included and streamlined, by assuring that all concepts appear and are therefore reported upon only once (direct re-usability of existing information).

It is a dynamic structure in the sense that additional statistical metadata and quality concepts can be included if necessary in the future.

The PDF icon first SIMS Manual was published in 2014, with a PDF icon revision in 2016.

Quality is of primary importance in the world of statistics and the implementation of SIMS supports the quality reporting on European Statistics.

Producers of official statistics have to guarantee that European statistics are developed, produced and disseminated on the basis of uniform standards and harmonised methods.

Furthermore, users of statistics are guaranteed access to appropriate metadata describing the quality of statistical outputs, so that they are able to interpret and use the statistics correctly.

ESMS (Euro SDMX Metadata Structure)

ESMS files are standardized, user-oriented files used for describing the statistical data sets published by Eurostat on its website.

The purpose of the ESMS files is to document the methodologies, the quality aspects, and the statistical production processes in general.

As the name indicates ESMS files are based on SDMX standards. The 19 high-level concepts are derived from the SDMX cross domain concepts published in the Word icon SDMX Glossary.

The ESMS standard is also increasingly used for reporting of national reference metadata files to Eurostat.

Below is an example of standardised reference metadata with both Eurostat metadata and National reference metadata.

ESMS-IP (Euro SDMX Metadata Structure – Indicator Profile)

The Excel icon ESMS-IP provides standard reference metadata for indicators.

The ESMS-IP is based on the Euro SDMX Metadata Structure (ESMS). In addition, it contains a short PDF iconquality box’ at the top providing users with a summary assessment of the main quality characteristics of an indicator.

It provides guidance for the use and the analysis of the indicator, and gives input to the selection processes of any new indicator sets. Examples of the use of the ESMS-IP standard are:

The ESMS-IPs are in particular used to document the quality of the Europe 2020 indicators and of the Sustainable Development Goals (SDGs) indicators. [After opening an individual indicator page (e.g. ‘No poverty’), click on the metadata icon icon to view the ESMS-IPs at the bottom of the page].

ESQRS (ESS Standard Quality Report Structure)

ESQRS is a standard for the production and dissemination of quality reports within the ESS.

ESQRS files provide users with detailed information for assessing the quality of the data sets released by Eurostat.

The broad concepts used are compatible with the SDMX cross-domain concepts and with the common terminology as published within the SDMX Glossary (2016).

This metadata structure is also increasingly used for national quality reports.

How to access ESMS and ESQRS files

ESMS, and ESQRS files can be found in Eurostat’s database. They are attached either to individual datasets or, more generally, to collection of datasets (based on similarity of data published). These files are indicated using the metadata icon icon.

The picture below shows examples of reference metadata files as attached to datasets or collections of datasets in Eurobase, Eurostat’s dissemination database for statistical datasets.

Example of reference metadata files in the data navigation tree

European Statistical System Metadata Handler (ESS MH)

The ESS MH is the web application developed by Eurostat for supporting the production, management, exchange and dissemination of European and national reference metadata files based on the above-mentioned standards.

The diagram below presents the high level business process for reporting SDMX compliant reference metadata and the usage of the ESS Metadata handler.

Diagram

(Edamis is the single entry point for all data and metadata entering Eurostat)

For more information about this application, see the ESS-MH dedicated page.

Further information