Statistics in development cooperation - quality in statistics

This article is part of a set of background articles providing an ‘outline of knowledge’ of international statistical cooperation between the European Union (EU) and developing countries, for non-statisticians needing to deal with statistics in development programmes and projects. The outline serves as an entry point and introduction to the much more detailed Eurostat publication 'Guide to statistics in European Commission development cooperation', downloadable in PDF format in English, French and Spanish (further down referred to as 'the Guide').

Many developing countries have weak statistical systems and mechanisms for measuring results. Good, reliable statistics are essential for measuring progress in reaching development goals and provide essential information about the effectiveness of policies and programmes. They help governments improve their policies and to be transparent and accountable about the delivery of development results. Reliable statistics are a key element towards better measurement, monitoring and management of the results of development assistance.

Statistical quality is most often defined as ‘fitness for use’ by end users. Quality therefore depends on data uses and users. Analysis of statistical quality permits the identification of target areas for capacity building. The ability to provide regular and reliable data on the economy and the well-being of the population is an important indicator of good policies and institutions.

Full article

Quality in statistics

According to the International Organization for Standardization (ISO), quality is:

“The totality of features and characteristics of a product or service that bear on its ability to satisfy stated or implied needs.”

Quality is not necessarily a synonym to absolute excellence: quality is often relative to the needs to be satisfied.

The key issues when evaluating the national statistical system (NSS) of a developing country:

Is the NSS capable of producing good statistics in terms of quality and quantity?
What statistics does the NSS actually produce?

This section is directly linked with the articles on data availability and on national statistical systems.

The dimensions of quality in statistics

Internationally adopted quality frameworks for statistics:

are used for assessing the quality of the data and of the procedures that are used in the data production;
consider all steps of the statistical process by which data are collected, transformed and disseminated.

Quality in statistics refers to the quality of the:

overall organisation of the process;
input data;
data collection, transformation and dissemination operations;
products (output data).

Guide examples and practical information

C.5.3 The concept of quality in statistics

What is essential for statistics of good quality?

The international community, through the United Nations Statistical Commission (UNSC), has agreed on ten ‘Fundamental principles of official statistics’ with which official statistics must comply to correctly inform the public.

The 10 fundamental principles of official statistics

Relevance, impartiality and equal access - Official statistics provide an indispensable element in the information system of a democratic society, serving the government, the economy and the public with data about the economic, demographic, social and environmental situation. To this end, official statistics that meet the test of practical utility are to be compiled and made available on an impartial basis by official statistical agencies to honour citizens' entitlement to public information.
Professionalism - To retain trust in official statistics, the statistical agencies need to decide according to strictly professional considerations, including scientific principles and professional ethics, on the methods and procedures for the collection, processing, storage and presentation of statistical data.
Accountability - To facilitate a correct interpretation of the data, the statistical agencies are to present information according to scientific standards on the sources, methods and procedures of the statistics.
Prevention of misuse - The statistical agencies are entitled to comment on erroneous interpretation and misuse of statistics.
Cost-effectiveness - Data for statistical purposes may be drawn from all types of sources, be they statistical surveys or administrative records. Statistical agencies are to choose the source with regard to quality, timeliness, costs and the burden on respondents.
Confidentiality - Individual data collected by statistical agencies for statistical compilation, whether they refer to natural or legal persons, are to be strictly confidential and used exclusively for statistical purposes.
Legislation - The laws, regulations and measures under which the statistical systems operate are to be made public.
National coordination – Coordination among statistical agencies within countries is essential to achieve consistency and efficiency in the statistical system.
International coordination - The use by statistical agencies in each country of international concepts, classifications and methods promotes the consistency and efficiency of statistical systems at all official levels.
International statistical cooperation - Bilateral and multilateral cooperation in statistics contributes to the improvement of systems of official statistics in all countries.

These fundamental principles form the basis for regionally adapted principles for official statistics.

Focus on: regional instruments which adapt the UNSC Fundamental principles of official statistics

Eurostat and the statistical authorities of the EU Member States adopted the European Statistics Code of Practice. This instrument:

represents the EU commitment towards high quality statistics;
builds upon a common European statistical system (ESS) definition of quality in statistics;
targets all relevant areas from the institutional environment, the statistical production processes to Eurostat output: European official statistics.

Eurostat supported the African Union to elaborate the African Charter on Statistics. This instrument:

was adopted in 2009;
awaits a sufficient number of ratifications to come into force:
- Advocacy for the ratification and implementation of this Charter at country level is a way to promote evidence-based decision-making and raising awareness for the importance of quality statistic;
- Two advocacy leaflets are available: Summary of the African Charter on Statistics; The African Charter on Statistics in seven questions.

In November 2011, the Statistical Conference of the Americas (SCA –ECLAC) adopted the Code of Good Practice in Statistics for Latin America and the Caribbean. It aims at being a technical and regulatory instrument contributing to the improvement of national statistical activities and systems across the region;

it is structured by 17 principles and associated best practices;
it has been inspired by the European Statistics Code of Practice;
it also addresses issues of specific relevance to the national statistical systems in the region: coordination of national statistical systems; cooperation and international participation;
for each of the 17 principles, compliance criteria have been defined;
a self-assessment questionnaire was developed for 10 principles of the Code of Practice
- assess the current status, strengths, weaknesses and improvement actions concerning the institutional environment and coordination, statistical processes and statistical outputs in the NSIs;
- the results have been presented in the Report on strengths, weaknesses and improvement actions found in the self assessment questionnaire measured against the principles of the Regional Code of Practice in Statistics for Latin America and the Caribbean.

Guide examples and practical information

B.2.1 Official statistics and their fundamental principles

Focus on: the United Nations Statistical Commission (UNSC)

The United Nations Statistical Commission (UNSC) is the apex entity of the global statistical system:

it brings together the chief statisticians from UN member states from around the world;
it is the highest decision making body for international statistical activities, especially the setting of statistical standards, the development of concepts and methods and their implementation at the national and international level;
it oversees the work of the United Nations Statistics Division (UNSD);
it is a Functional Commission of the UN Economic and Social Council.

How to assess the quality of statistics?

In general, the tools used to promote and support quality in statistics are:

charters or other instruments which define the principles of statistics quality for a region
quality requirements in national statistical legislations
national quality commitments and development of national quality frameworks
regular monitoring and reporting on implementation of and compliance with these quality frameworks
assessment questionnaires, whether for self-assessment and / or peer reviews

The European statistical system's experience

To implement the European Statistics Code of Practice, the European statistical system:

assesses the progress of the implementation of the Code of Practice;
devised a self-assessment questionnaire;
systematically carries out peer reviews;
based on the peer reviews and complementary information, in 2008 Eurostat reported to the European Parliament and the Council on the implementation of the Code of Practice in the European statistical system
- improvement actions defined by the individual national statistical institutes and Eurostat;
each year, Eurostat monitors compliance with the Code of Practice across the ESS
- summary information is provided to the European Statistical Governance Advisory Board (ESGAB) for its Annual Report to the European Parliament and the Council.

Available information on the European experience on quality in statistics:

Eurostat: Definition of quality in statistics (2003)
Eurostat: Handbook on Data Quality Assessment Methods and Tools (2007)
Eurostat webpages on quality
Guide section C.5.3.1 The key quality concepts in statistics

International tools for assessing quality in statistics

Eurostat is currently developing a new tool for EU delegations to create a ‘Snapshot’ of national statistical systems and their capabilities:

to identify the main strengths and weaknesses of national statistical systems;
to assess their ability to produce reliable data;
to create a basis for dialogue with the government on how to improve the quality of the system.

This 'Snapshot’ tool comprises 2 modules:

a diagnostic of the national statistical system as a whole, focusing on the statistical infrastructure and organization;
a method to assess data quality in specific sectors and data quality of key sector indictors (those used in the performance assessment framework).

Eurostat: The 'National Statistical Offices' questionnaire was developed during 2011 to collect information about National Statistical Offices in African, Caribbean and Pacific (ACP) countries

based on the information available in the United Nations Handbook of Statistical Organizations;
also based on a previous pilot questionnaire developed for Asian and Latin American (ALA) countries;
tested with PALOP countries (Portuguese-speaking African countries);

It investigates and documents 8 key aspects:

general information on the NSI;
NSI budget and financing;
activities of the NSI;
the national statistical system;
dissemination of official statistics;
the National Strategy for the Development of Statistics (NSDS);
future projects and expertise of the NSI;
comments and suggestions from the NSI.

International Monetary Fund: General Data Dissemination System (GDDS)

is a framework to develop national statistical systems in producing economic and socio-demographic data;
gives countries a management approach that fosters sound statistical methodology, professional data compilation, and data dissemination practices;
diagnoses data quality issues and catalogues technical assistance needs, guiding the GDDS users (incl. national policy makers and donors) on how to address these issues;
serves as a vehicle for countries to gradually progress and graduate to the special data dissemination standard (SDDS)
- GDDS Factsheet.

International Monetary Fund: Data Quality Assessment Framework (DQAF)

identifies quality-related features of governance of statistical systems, statistical processes, and statistical products;
is rooted in the UN Fundamental Principles of Official Statistics and grew out of the Special Data Dissemination Standard (SDDS) and General Data Dissemination Dystem (GDDS), the IMF's initiatives on data dissemination
- DQAF Factsheet.

Organisation for Economic Co-operation and Development: Quality Framework for OECD Statistical Activities

International Organization for Standardization: ISO 9000 quality standard

Guide examples and practical information

C.5.2.3 Assessment methodologies
Box 5.4: A new tool for EU delegations to create a ‘Snapshot’ of National Statistical Systems and their capabilities
Box 5.5: Eurostat National Statistical Offices questionnaire for African, Caribbean and Pacific countries
Box 5.6: Case study: Assessment of the statistics systems in Central Asia
Box 5.7: Country Statistical Situation Reports in MEDSTAT
Box 5.9: The General Data Dissemination System (GDDS)
Box 5.10: The Data Quality Assessment Framework (DQAF)

Statistics in development cooperation - quality in statistics

Full article

Quality in statistics

What is essential for statistics of good quality?

How to assess the quality of statistics?

Direct access to