Eurostat, the statistical office of the European Union.

E1: Agriculture and fisheries

2920 Luxembourg LUXEMBOURG

Animal production statistics cover three main sub-domains based on three pieces of relevant legislation and related gentlemen’s agreements.

• Livestock and meat statistics are collected under Regulation (EC) No 1165/2008. They cover meat production, as activity of slaughterhouses (monthly) and as other slaughtering (annual), meat production (gross indigenous production) forecast (semi-annual or quarterly), livestock statistics, including regional statistics. A quality report is also collected every third year.
• Milk and milk product statistics are collected under Decision 97/80/EC implementing Directive 96/16/EC. They cover farm production and utilisation of milk (annual), collection (monthly for cows’ milk) and production activity by dairies (annual) and statistics on the structure of dairies (every third year). An annual methodological report is also collected.
• Statistics on eggs for hatching and farmyard poultry chicks are collected under Regulation (EC) No 617/2008, implementing Regulation (EC) No 1234/2007 (Single CMO Regulation). They cover statistics on the structure (annual) and the activity (monthly) of hatcheries as well as reports on the external trade of chicks.

European Economic Area countries (EEA - Iceland, Liechtenstein and Norway) are requested to provide milk statistics, with the exception of those related to home consumption, as stated in Annex XXI of the EEA Agreement. Liechtenstein is exempted in the Agreement, only Norway is concerned.

The Agreement between the European Community and the Swiss Confederation on cooperation in the field of statistics states that Switzerland must provide Eurostat with national milk statistics and, after 2013, livestock and meat statistics.

The same statistics are requested from the candidate and potential candidate countries as EU acquis.

The statistical tables disseminated by Eurostat are organised, under Animal production (apro_anip), into three groups of tables on Milk and milk products (apro_mk), Livestock and meat (apro_mt) and Poultry farming (apro_ec). This later label covers statistics on hatcheries and trade in chicks and on production of eggs for consumption. The regional animal production statistics collected on livestock (agr_r_animal) and on cows’ milk production on farms (agr_r_milk_pr) are disseminated separately.

Due to the change in the legal basis or in the methodology, the time series may be broken. This is indicated by a flag in the tables.

Further data about the same topics refer to repealed legal acts or agreements. The tables on statistics on the structure of rearing (apro_mt_str) and the number of laying hens (apro_ec_lshen) or of equidae (apro_mt_lsequi) are therefore no longer updated. The same applies to some variables (external trade of animals and meat), periods (surveys in April or August) or items (number of horses) included in other tables.

The detailed content of each table and the reference to its legal definition is provided in the table below.

Table 3.1: Data tables disseminated regarding animal production statistics

Some standard code lists cover the concepts over the domains. The list of items and their definition is in any case derived from the legislation, but the coding integrates different approaches. The following concepts have been integrated in a single list agriprod:

• Live animals
• Meat and meat products
• Dairy and other animal products (except meat)

A handbook on concepts and definitions used for animal production statistics is provided in Annex 1.

Explicit size classes are used to describe the dairy entreprises by yearly quantities (t) collected, processed or produced and the hatcheries (egg incubation capacity).

Regional data

The territorial classification of regional data (tables agr_r_milkpr and agr_r_animal) is broken down according to the NUTS classification for Member States and to Eurostat’s list of Statistical Regions for Candidate countries and EFTA countries.

Statistics on livestock, on farm production, on other slaughtering and utilisation of milk cover agricultural holdings in Member States.
The minimal coverage for livestock sample surveys is of at least 95 % of the national population with reference to the last survey on the structure of agricultural holdings (FSS).

Livestock surveys may be conducted independently by livestock category or as a sub-set of items surveyed with a wider scope (livestock survey as a whole, farm production survey, annual census) or recorded with a wider objective in the case of registers (every animal owner). Depending on the design, some over-coverage can be observed.

Statistics on external trade in chicks are designed to reflect foreign trade in chicks from hatcheries with more than 1,000 incubation places.

Other animal production statistics cover EU plants whose activity is slaughtering, milk collection, milk product production or hatching fertilised eggs. The surveys must cover exhaustively dairies representing 95 % of the cows’ milk collected by Member States, the balance being represented by sampling or other sources. The methodological questionnaire collected from Member States reports whether a correction is applied for cross-border collection of milk from dairy farms in a country by dairy enterprise in a neighbour country.

See also Annex 1 for further explanations.

Among concepts used in animal production statistics (see Annex 1), some can be reported because of their specificity.

Gross indigenous production (GIP) is the number of animals slaughtered plus the balance of intra-Community and external trade for the same kind of live animals. GIP is thus the number of animals from a Member State (indigenous) apparently (gross) slaughtered or exported alive.

Slaughtering is measured through activity of slaughterhouses from 1 January 2009 (application of Regulation (EC) No 1165/2009), i.e. production of marketable meat for human consumption. Estimates of ‘other slaughtering’ can be added for a more accurate picture of meat production.

Livestock population is accounted by categories that capture their rearing, either for fattening then slaughter, or for herd renewal, i.e. for breeding and/or milking.

Milk statistics are led by the concept of ‘national dairy’, i.e. the dairy sector is considered as a single process, which internal flows are not (intended to be) taken into account.

Use of raw milk is followed through production of its two main components, fat and protein content. Milk processed is thus accounted for as an aggregate of UWM (utilised whole milk, with the full content of fat and proteins) and USM (utilised skimmed milk, with the full content of proteins,without fat). As a process can produce skimmed milk further to the main (fat) product and, in such a case, USM can be negative. This is especially the case for cream and butter production.

Bovine animals are domestic animals of the species Bos taurus and Bubalus bubalis, including hybrids like Beefalo. Clarification on the implementation of this definition led to the integration of buffaloes and hybrids into the category 'bovine animals'.

Chickens means all animals of species Gallus gallus, including broilers and boiling hens. This concept was applied with Regulation (EC) No 1165/2008 on 1 January 2009.

Regional data

Region means a sub-division of a Member State territory. Depending on the statistics, 'region' refers to:

• NUTS 2 for milk production (table agr_r_milkpr), the NUTS reference being the version applicable on the date of data transmission.
• NUTS 2, except for DE (NUTS1) for livestock statistics (table agr_r_animal).

For data on the structure of hatcheries, a particular region (the most important) can be considered as representative of the national data in BG, EL, LV and AT.

Agricultural holdings are the statistical units for livestock surveys and animal production statistics at farm level (milk production and meat). Depending on the statistics collected, a more precise definition can be used, based on their activity or their structure, e.g. dairy farms producing raw milk, or farms with livestock or with sheep or goats.

Dairy enterprises -- undertakings of two types:

• Collection centres collect milk or cream and transfer it in whole or in part to other enterprises without any processing. They are often defined as referred to in Article 2(2) of Council Directive 96/16/EC.
• Dairies and agricultural holdings purchase milk or milk products from agricultural holdings or collection centres with a view to transforming them into milk products. They are often defined as referred to in Article 2(1) of Council Directive 96/16/EC.

Some enterprises process milk products obtained from a dairy as defined above, e.g. skimmed milk into milk powder or yogurt, and may appear to be excluded from the definition of dairy enterprises. Nevertheless, non-packed intermediate products are considered as raw products and such dairies are therefore covered as statistical units for the purpose of some statistics, whatever the enterprise supplying them.

Slaughterhouses are registered and approved establishments used for slaughtering and dressing animals whose meat is intended for human consumption. In countries in which ‘hygiene package’ is not fully implemented (slaughterhouses not registered or not approved by the EU can nevertheless produce for the local market) all slaughterhouses are covered.

Hatcheries are establishments for incubating eggs, hatching and supplying chicks (exceptionally almost hatched eggs).

For some animal production statistics, the statistical units are not explicitly defined, i.e. they refer to all enterprises. This is the case for the statistics on trade of chicks and this used to be the case for slaughtering statistics. Indirectly, the statistical units are the reporting enterprises dealing with one of parameters to be measured by the statistics.

The statistical population is the framework of the statistical units in the reference Member State or country for the reference period.

Nevertheless, data collection may be organised in a different way by a respondent other than the statistical unit. For instance, milk delivered by farms to dairies is accounted for by both units, and can be obtained more easily from dairies, of which there are fewer.

For monthly milk statistics, the population covers the dairies collecting cows’ milk. The quantity of milk products processed may be therefore underestimated compared to national production.

The reference area is the territory of the Member States as defined by Decision 91/450/EEC. For non-EU countries, territory follows the definition agreed bi-laterally with Eurostat.

Data are presented in chronological series for each country and for the EU. The covered period varies according to the country, depending on the date of accession to the EU or of application of the legal requirements for statistics. They are available at least for the EU Member States, depending on the date they became members of the EEC or EU, from legislation to repeal of the legal text and insofar as Member States comply with legislation.

Statistics are available, at least partly, since reference years 1960 (milk statistics), 1964 (meat statistics) and 1970 (livestock statistics and statistics on eggs for hatching and chicks) for the countries that were Member States at this time.

Further time periods may be available if collected on a volunteer basis, especially:

• Data collected under a gentlemen’s agreement
• Data collected for accessing and acceding countries
• Data collected under other specific agreements

The production forecast cover four or six quarters for pigs, three or four semesters for bovine animals, and two or zero semester for sheep and goat, depending on the size of the national livestock. At the end of September, year N, the forecast must be available up to the end of: - year N+1 for bovine animals and - first half of year N+1 for pigs. At the end of February, the forecast regarding sheep and goats must be available up to the end of the year for the countries with significant production.

Not applicable.

The units of measure are:

For non-cumulative variables such as stock or inventory variables, the reference date is

• a given day during the period displayed in ‘Month’ for livestock (tables such as apro_mt_ls*),
• on 31 December of the year indicated in ‘Time’ for annual and less frequent data (apro_ec_lshen, apro_ec_strpoul, apro_ec_strhen, tables like apro_mk_str* and like apro_mt_str*).

For cumulative variables such as production or flows, the reference periods are:

• the calendar month for data in tables apro_mk_colm apro_mt_pwgtm, apro_mt_pheadm and apro_ec_poulm,
• the calendar quarter for GIP forecast for pigs in table apro_mt_ppighq (pig production forecast),
• the calendar semester for other GIP forecast in table apro_mt_pcatlhs (bovine, sheep and goat production forecast),
• the calendar year for all other animal production statistics.

The legal acts currently applicable are:

• Regulation (EC) No 1165/2008 of the European Parliament and of the Council of 19 November 2008 concerning livestock and meat statistics and repealing Council Directives 93/23/EEC, 93/24/EEC and 93/25/EEC. OJ L 321, 1.12.2008, p. 1–13.
  Germany and Bulgaria were granted derogations from the application of this Regulation until 1 January 2010 or, in the case of statistics on sheep and goats, until 1 January 2011 (Commission Decision 2010/323/EU).
• Directive 96/16/EC of 19 March 1996 on statistical surveys of milk and milk products. OJ L 78, 28.3.1996, p. 27–29.
• Decision 97/80/EC of 18 December 1996 laying down provisions for the implementation of Council Directive 96/16/EC on statistical surveys of milk and milk products. OJ L 24, 25.1.1997, p. 26–49.
• Commission Regulation (EC) No 617/2008 of 27 June 2008 laying down detailed rules for eggs for hatching and farmyard poultry chicks (OJ L 168, 28.6.2008, p. 5–16) implementing Regulation (EC) No 1234/2007.

The following agreements are also applicable:

• Agreement on the European Economic Area (OJ No L 1, 3.1.1994, p. 3; and EFTA States’ official gazettes), Annex XXI, regarding milk statistics
• Agreement between the European Community and the Swiss Confederation on cooperation in the field of statistics

Finally some former legal texts and agreements were previously applied. When feasible, Eurostat ‘aligns’ the time series, possibly with an indication for breaks (flag ‘b’); otherwise, the series looks incomplete. In some cases, entire tables are no longer updated.

• 2014 CPSA agreement on estimates of monthly other slaughtering
• 2017 ESS Agreement on statistics of eggs for consumption

From reference year 2025 onwards, Regulation (EU) 2022/2379 of the European Parliament and of the Council of 23 November 2022 on statistics on agricultural input and output will replace the above legislations and agreements.

Regional data

Regulation (EC) No 1059/2003 of the European Parliament and of the Council of 26 May 2003 on the establishment of a common classification of territorial units for statistics (NUTS).  OJ L 154, 21.6.2003, p. 1-41;. 

Not applicable.

Regulation (EC) No 223/2009 on European statistics (recital 24 and Article 20(4)) of 11 March 2009 (OJ L 87, p. 164), stipulates the need to establish common principles and guidelines ensuring the confidentiality of data used for the production of European statistics and the access to those confidential data with due account for technical developments and the requirements of users in a democratic society.

A confidentiality charter for animal production statistics has been adopted in 2016 (below) and should lead to disseminate more EU totals from 2020 on.

The confidential data are identified as such by the Member States, based on their national rules to protect confidential information. Those data are protected from disclosure by agreed procedure. Where no more sophisticated rule is applied, any value derived from a confidential value is considered as confidential. The confidentiality charter (see 7.1.) is progressively implemented on the Animal Production Statistics data.

Data are disseminated in the Eurostat public database as soon as they are available.

The release calendar indicates when the data users can expect access all the due data.

See item 8.1.

In line with the Community legal framework and the European Statistics Code of Practice Eurostat disseminates European statistics on Eurostat's website (see item 10 - 'Accessibility and clarity') respecting professional independence and in an objective, professional and transparent manner in which all users are treated equitably. The detailed arrangements are governed by the Eurostat protocol on impartial access to Eurostat data for users.

Data dissemination usually follows data collection by a few days. Frequency is thus the same in both cases for a given data set.
For most of the data collected, collection frequency is the time granularity of the data, i.e. monthly data are collected monthly and annual data annually.
Annual totals provided in tables apro_mk_cola, apro_mt_pann and apro_ec_poula are disseminated at the same time as data for December.
GIP forecasts for meat are collected twice a year or, for countries with a limited livestock population, annually.
Data on the structure of dairy enterprises are collected every third year.

News releases on-line.

Publications

Publications available in pdf format are accessible on Eurostat's website (Statistics / Agriculture and fisheries / Agriculture / Publications). Eurostat has published the Eurostat regional yearbook 2022.

The Directorate General on Agriculture and Rural Development (DG-AGRI) publishes regularly a Short-term outlook.

Statistics Explained

Statistics Explained is an official Eurostat website presenting many statistical topics in an easily understandable way. Together, the articles make up an encyclopaedia of European statistics, completed by a statistical glossary clarifying all terms used and by numerous links to further information and the very latest data and metadata, a portal for occasional and regular users alike.

Animal production articles:

• Milk and milk product statistics
• Agricultural production - livestock and meat
• Poultry statistics

Articles referring to animal production statistics

• Agriculture statistics at regional level: Livestock and milk

Re-dissemination

The Eurostat statistics are widely re-disseminated. For improving the market transparency, the EU Milk Market Observatory provides for instance statistics on milk production and the EU Meat Market Observatory statistics on meat production or livestock.

The data on animal production statistics and statistics for regional production may be found on the Eurostat’s dissemination database under the link: https://ec.europa.eu/eurostat/data/database and under the following paths: Database / (+) Data navigation tree / (+) Database by themes / (+) Agriculture, forestry and fisheries / (+) Agriculture (agr) / (+) Agricultural production (apro) / (+) Animal production (apro_anip) / (+) Poultry farming (apro_ec), or (+) Milk and milk products (apro_mk), or (+) Livestock and meat (apro_mt).

This channel displays all the public Animal Production results.

Not applicable.

No other dissemination format.

Public documentation on methodology and legislation is available on-line.

Member States provide Eurostat with a report on the quality of the data transmitted every third year under the current legislation on livestock and meat, and for the first time for reference year 2010.

Member States send Eurostat an annual methodological questionnaire on milk statistics providing some quality indicators.

This information may be disseminated to the public once this is agreed with Member States.

The specific EU metadata files on livestock and meat statistics and on statistics of milk and milk products (see the links at the bottom of the present file) summarise the national quality reports and provide the direct links to them.

Animal production statistics are subject to the general quality assurance framework of Eurostat, where domain-specific quality assurance activities (the use of best practices, quality reviews, self-assessments, compliance monitoring) are carried out systematically.

 The latest quality review should provide findings and recommendations in 2019, for the whole process of all Eurostat animal production statistics. The following were recommended.

1.1) A legal basis for monthly “Slaughtering other than in slaughterhouses” from annual to monthly.
1.2) A legal bass for the “Eggs for human consumption”.
2.1) A release calendar for the Animal production data.
3.1) Metadata with more detailed information and maintain the metadata up-to-date.
3.2) Final step-by-step process description.
3.3) Improved particular quality concepts for Livestock and meat statistics.  
3.4) New national quality reports for poultry statistics and summary (EU) quality report.
3.5) New applicable Quality and performance Indicators and public existing QPIs.
3.6) Additional or improved handbooks, guidelines, manuals, especially handbooks and manuals for Meat and livestock statistics and Poultry statistics.
4.1) Statistics Explained article for Poultry statistics.
4.2) Availabledata on “Eggs for consumption” in Eurobase.
5.1) Validation rules for poultry statistics, shared with countries.
5.2) Additional validation rules for Animal Production statistics, at national and Eurostat level. 
6.1). EU aggregates based on already agreed methods [i.e. confidentiality charter].
7.1) Docuiment on the breaks in time series and comparability of data over time.

The initial and subsequent actions were conducted but, depending on the duration of the whole procedures, the public outcomes are not yet all available.

For instance, establishing a new legislation / agree with Member States, collect, validate, analyse, disseminate national quality reports / develop, test, implement IT adhoc procedures require years.

The main users are other Directorates General of the European Commission (e.g. DG Agriculture and rural development and DG Health). However, there are other major users such as other European institutions, national administration services, national statistical institutes, other international organisations, agro-industry, producer groups, research institutes, journalists, third countries and the public in general. The objectives of these users vary, but animal production statistics are especially useful for market management/monitoring, production forecasts and policy-making in agriculture and food.

The user needs were re-assessed when drafting the Strategy on agricultural statistics for 2020 and beyond andthe subsequent EU legislation.

Key users are well known and their needs are met. In addition, specific questions from individual users are answered.
Eurostat conducts regular user surveys with a wider scope.

Since 2010, completeness has been measured regularly and discussed during meetings of the Working Party on Annual Production Statistics. Meanwhile it has been regularly improved.

To reduce the burden on respondents, some statistics are collected less often in countries with limited importance in the EU totals. Statistics on bovine population are due only once a year for the Member States where it is below 1.5 million head. Statistics on pig population are due only once a year for the Member States where it is below 3 million head. Statistics on sheep population are not expected from the Member States where it is below 500 000 head. Statistics on sheep population are not expected fromthe Member States where it is below 500 000 head.

Therefore, some data are not due and may appear to be missing, due to the ‘cube’ effect of the selection tools in the Eurostat dissemination database (each selected code of a dimension is crossed with each selected code of each other dimension).

Regional livestock population is not mandatory when it is under 75 000 bovine animals, 150 000 pigs, 100 000 sheep and 25 000 goats.

Accuracy is normally accuracy displayed and reflects accuracy of computation. Nevertheless threshold for significance is half of the displayed unit. It means that '0.000' with flag 'n' is lower than 0.5 whereas 0.001 is lower than 0.0005. '0.000' without flag 'n' is a true zero, i.e. with full accuracy.

Regulation (EC) No 1165/2008 states that sampling errors for the results of each Member State shall not exceed (with a confidence interval of 68 %):

• 1 % of the total number of bovine animals (5 % where the bovine animal population is below 1 000 000 head);
• 1.5 % of the total number of cows (5 % where the cow population is below 500 000 head);
• 2 % of the total number of pigs (5 % where the pig population is below 1 000 000 head); and
• 2 % of the total number of sheep and goats (5 % where the sheep and goat population is below 1 000 000 head).

The Member States report to Eurostat on the actual accuracy but these national figures are not publishable.

Directive 96/16/EC states that the Member States may carry out the milk monthly surveys in the form of sample surveys. In that case, the sampling error must not exceed 1 % of the total national [milk] collection (with a confidence interval of 68 %).

No quality criterion is defined in Regulation (EC) No 617/2008 (market Regulation).

Measurement errors are due mainly to lack of harmonisation in statistical methods. For instance, when EU concepts do not fit with national concepts, there may be significant measurement errors. As well over-coverage (surveying units out of the statistical population) may be due to the same, when the harmonised definition of the statistical unit is not implemented. Data providers are expected to correct these deviations.

Table 3.1. (see 3.1. data description) shows the deadlines for the various animal production statistical tables, given relative to the end of the reference period.
The actual timeliness (length of time between the event and availability of the statistical output) can be shorter than the legal timeliness if data are provided earlier. The time lag between the actual release date and the planned (agreed or legal) date is called punctuality.
The actual timeliness for EU results depends on timeliness achieved among Member States. Time taken for data validation and dissemination by Eurostat is also taken into account.

Data are normally received within the legal deadlines. However, some countries may experience delays in sending their data to Eurostat due to exceptional circumstances.

The below attached document on punctuality intends to summarise punctuality of the Member States without disclosing the bilateral issues between Eurostat and the Member States. Punctuality of the EU results is, for each value, punctuality of the latest Member State. 

Animal production statistics are mostly comparable over countries and regions with the following exceptions:

• The ESS and CPSA agreements may be granted with only some countries
• A derogation to EU statistical legislation may have been granted
• The legislation itself provides for differentiated approaches for countries depending on their livestock population (see 12.3 Completeness)
• Specific agreements with EEA countries and with the Swiss Confederation envisage limit application of the legislation and exceptions to the definitions

Other non-EU countries may partly implement EU legislation and some concepts may be irrelevant for them. Quality of the statistics is usually not at the same level as for statistics under EU legislation.

Comparability of regional data over time has been and will be affected by changes in the NUTS classification.

Animal production statistics are largely comparable over time, with the following exceptions:

• Implementation of Regulation (EC) No 1165/2008 (see ASA-TE-696.Inventory_of_changes.rev1.doc )
• Change in NUTS for regional statistics
• Other particular changes in the methodology flagged with "b", indicating breaks in the time series

Major revision may downgrade comparability over time or, when they intend to be corrective, improve it (below attached document)

Most livestock categories are comparable with certain characteristics from the farm structure survey, but differences in the design of each survey (especially reference date and calibration method) may produce discrepancies in the estimates.

Milk products are comparable with the Prodcom products, but the recorded flows are different (added value for Prodcom, black box for milk statistics).

Internal coherence is insured through pre-validation checks during the data collection (for the same country and the same period) and validation on the data received in Eurostat (time series).

Permanent efforts are done to limit the burden on the respondents and, if relevant on the competent national authorities.

• The design of Regulation (EC) No 1165/2008 makes optional the delivery of some data sets for the countries under a certain threshold. This approach by differentiate intensity of burden relatively to the policy interest is favoured in the new data collection (gentlemen's agreements). This might make difficult using the tables with national values, due to the gaps.
• Use of administrative sources is favoured (livestock statistics) when it is not the prefered data source (slaughtering, eggs hor hatching).
• Involvement of the respondents as data users (milk statistics) contributes to a balanced burden.
• The supply balance sheets, requested previously at national level are now compiled , with few exceptions, at EU level
• The Strategy on Agricultural Statistics for 2020 and beyond provides an opportunity to review the statistical needs.

Data are revised if figures of higher quality become available.

In order to provide fresh statistics, Regulation (EC) No 1165/2008 foresees that provisional livestock statistics are delivered first (in February and September) and definitive results are provided later (in May and October).

Results can be corrected by the countries concerned during the months after data has been sent.

For annual milk statistics, after a first validation round, the values which does not return error are published as provisional. Advanced validation is then run, leading either to confirmation or to revision of these values.

Revisions of historical data or of long time series are exceptional and the most recent have been documented (see 15.2. Comparability over time).

The data sources may be sample surveys or censuses for milk and livestock statistics. Nevertheless administrative sources may be used for obtaining these results in order to limit burden on the respondents. This is especially the case for bovine livestock. The milk quota registers have been used up to the end of the milk quota regime (April 2015).

See also Annex 3 for livestock and meat statistics.

For most of the data collected, collection frequency is the time granularity of the data, i.e. monthly data are collected monthly and annual data annually.

Annual totals provided in tables apro_mk_cola, apro_mt_pann and apro_ec_poula are disseminated at the same time as the data from December.

GIP forecast for meat are collected twice a year or, for the countries with a limited livestock population, annually.

Data on the structure of dairy enterprises are collected every third year.

Data and metadata transmission is executed in the same manner in all three sub-domains. The input consists of the national data which are ready for transmission and the output comprises the CSV data files in EDAMIS (the ESS data transmission system) servers.

CNAs transmit regional (where required) and national statistics to Eurostat exclusively via EDAMIS. The data comprise all variables listed in Chapter 2 heading 5 except those for which it is indicated that they are computed by Eurostat. The data comprise five datasets on milk statistics, 16 datasets on livestock and meat production statistics and five datasets on poultry statistics, displayed in Table 2. Special notes about the transmission of some livestock and meat datasets are given in Table 3.

The transmission means used is EDAMIS with MSExcel forms. A specific form exists for each dataset, presented as a spreadsheet table. The form sent through EDAMIS is checked for its structure and content and the data sender is notified of the outcome with reference to the error report, if any.

If there is no error and no non-explained warning the data are delivered by EDAMIS as CSV files and loaded by the production system.

The domain manager is automatically notified by EDAMIS on the file becoming.

Confidential data

Each data item is accompanied by a flag which shows whether the data is confidential or not.

Regarding milk statistics in particular, 14 countries flag certain items as confidential since three or less dairy enterprises had a significant contribution to the production of milk or of milk products such as powered dairy products.

Monitoring of data transmission

The monitoring of data transmission starts at the time the collection takes place. Countries which have not provided data in time are notified with a reminder automatically sent by EDAMIS. The Eurostat domain managers contact the data provider if the automatic reminders are without effect. In case of non-response Eurostat contacts the data provider again before sending the formal reminder letter, which is the last step before starting an infringement procedure.

1. Data validation in Member States

Data validation in Member States is known depending on their reporting.

• For milk statistics, Member States validate the data collected from farms and dairies. Comparisons with data obtained from the same dairy/farm in previous collection rounds of the survey and with data from other surveys are usually carried out. Moreover, variables collected from dairies are compared with similar ones collected from farms. No more information about other validation performed by the Member States before data submission to Eurostat is available.
• For livestock and meat statistics, the information about data validation is scarce. It covers all MSs with the exception of Austria since at the time of writing the present document no quality report was received from the country.
  Eleven CNAs report that they implement “real-time” validation of the data they collect, i.e. validation at the moment of collection. 22 CNAs mention explicitly that they validate the data: 18 compare the data with data for the same statistical units from earlier data collections, while 14 CNAs do other comparisons, which are not described. One CNA uses the results of a sample survey to check the data about slaughtering collected monthly from administrative sources.
  Reporting of the use of imputation, re-calibration or other correction / adjustment methods is connected explicitly with the treatment of non-response. It cannot therefore be known whether they are also used to correct other errors identified during validation. Re-calibration is used by eight CNAs, to treat non-response in livestock surveys. Imputation is employed by four CNAs in the case of monthly slaughtering statistics and by 14 CNAs for livestock statistics. The most usual imputation method is to take data for the same statistical unit from the previous data collection. A small number of CNAs use donor imputation or take data from other surveys or registers.
• On eggs for hatching and chicks, no information is available about data validation performed by the Member States before data submission to Eurostat.

2. Validation rules agreed with Member States

The validation rules agreed with Member States are documented in a Handbook on data validation in animal production statistics.

The validation automatically implemented by EDAMIS are of three severity levels:

- Error: a validation rule which must be satisfied. As long as the form contains errors it is rejected;

- Warning: a validation rule which can be broken. The data sender is required to explain why the rule is broken so that the file is acceptable;

- Information: a rule which provides feedback. The rule does not lead to reject the file, but its outcome is reported in the error report.

Validation of animal production statistics
Validation, level 0
After checking that the file structure is compliant with the expected one, it is checked that all mandatory cells contain information (data or flags) and that data types are valid. The flags that can be inserted in the files are pre-defined and available to the data senders as a drop-down list.
All data cells of all transmitted datasets are mandatory with few exceptions:

• All data cells for regional data of countries with a single region,
• Livestock statistics for the regions having fewer than 75 000 bovines, 150 000 pigs, 100 000 sheep or 25 000 goats, if these regions represent 5% or less of the national population of the relevant animals,
• Data cells regarding optional items (Milk table B, ad hoc aggregates like 'adult bovines' or sub-aggregates for the dairy structure)

Validation, level 1
Validation of level 1 includes numerous rules specifying the allowed range of values for data cells and also certain rules for consistency between cells (mainly totals and sums of components).

3. Data validation - detection (Eurostat)

The first processing step is to validate the data and load them in the production system. The input to this process are the datasets received and the output are either the data loaded or validation reports and requests to the CNAs for feedback or revised data.

It is reminded to the reader that during data transmission automatic validation was performed (see Data collection).

Data files are loaded and the Eurostat users can review the available information and the comments made by the CNAs and they can combine data from the available input files and create ad hoc data tables. Automated and non-automated validation is carried out with the tool.

Validation, level 2 - detection of outliers

The automated validation is the detection of outliers in the received data. Time series are created by retrieving data from earlier reference periods. Each current value is considered aa a possible outlier when it is outside the interval (Q1 – 1.5*(Q3-Q1), Q3 + 1.5* (Q3-Q1)), where Q1 and Q3 denote the first and third quartiles respectively of the corresponding time series. This validation is applied to all cells of the received datasets. Outliers are not necessarily wrong data.

Validation, level 1 - consistency in the data file

Eurostat may implement additional validation rules to check the consistency of the outliers with current data from the same data file. The following types of additional checks are used:

- Aggregation of items: data items that represent category totals (e.g. total bovine animals) must be equal to the sum of the items representing the different categories, allowing for certain tolerance limits.

- Aggregation of regions: data items that represent regional totals (e.g. totals at country level) must be equal to the sum of items representing the corresponding sub-regions.

- Consistency of totals and partial components: sometimes not all categories corresponding to a particular total are collected (e.g. cows’ milk part of which is obtained from dairy cows’ milk, in Table C). In these cases the data items that represent category totals must be greater than or equal to the sum of the items representing the different categories.

- Sign of the values: all variables should be non-negative. Some variables however regarding milk production in Tables B and C are allowed to receive negative values (e.g. in Table B-Milk production, the variable ‘Input of skimmed milk (USM)’ can be by definition negative in case that the process produces skimmed milk (e.g. cream processing)).

- Consistency by size class: the average size class of milk collected and volume of milk treated or milk production in dairies enterprises is computed by size category and is checked whether it falls in specified ranges.

Validation, level 3 - comparison with other data sets

Eurostat may implement additional validation rules to check the consistency with related datasets. The following types of additional checks are used:

- Correspondence between variables from different data sets: same or similar variables from different datasets are compared (e.g. the quantities of products obtained in Table B-Milk Production should be equal to the ones reported in Table H-Milk Protein Contents).

- Consistency between variables from different data sets: related variables in different tables must be consistent. For example if Quantity of utilized product in Table B-Milk Production is zero (plus tolerance limits) then the Protein content of the same product in Table H-Milk Protein content must be less than 1000 (plus tolerance).

These checks are additional to the consistency checks already implemented in eWF. Since they are not applied to all data items they are not systematic and moreover they are not automated; the user chooses on which items to apply them and which rules to apply, based on intuition.

At the end of this step, validated data are forwarded for subsequent processing and production of EU statistics.

4. Data validation - correction (Eurostat)

Depending on the results of validation Eurostat may decide to reject the dataset or to request feedback from the CNA postponing loading in the production system. In either case it prepares an email to the registered data provider with a report of the validation findings. 

• Calculation of results for country aggregates (e.g. EU-28, EU-27, etc.).
• Calculation of annual totals based on monthly values
• Calculation of item aggregates
• In table apro_ec_poulm, derived series are calculated from the primary data; from these it is possible to estimate numbers of birds and production (eggs) by applying coefficients.
• Statistics on trade of chicks are drawn from Intrastat/Extrastat (Comext) for six Member States and under their responsibility
• On regional statistics, imputation of values to equivalent regions, collected only once (e.g. LU, LU0, LU00)

None.

The current description covers decades of methodology on statistics. It reflects especially the current design of animal production statistics. Usability of the data promotes continuity of the time series, which may downgrade interpretability of the statistics, especially considering the permanent improvement in the statistical production process. Eurostat expects having reached a satisfying trade-off. The data user is nevertheless invited referring to the relevant legislation applicable at the reference date for more accurate analysis involving long time series.