6. Accuracy and reliability |
Top |
|
not applicable |
6.1. Accuracy - overall |
Main sources of error |
The following are considered to be the main sources of error in FSS2016:
- coverage error,
- non-response error,
- measurement error.
|
|
6.2. Sampling error |
Method used for estimation of relative standard errors (RSEs) |
See Annex. |
Annexes: 6.2. FSS 2016 variance estimation |
6.2.1. Sampling error - indicators |
1. Relative standard errors (RSEs) - in annex
2. Reasons for possible cases where precision requirements are applicable and estimated RSEs are above the thresholds |
There are no cases where the estimated RSEs are above the thresholds where precision requirments are applicable. Crop and livestock characteristics are collected for each holding in the population. The RSEs have been however calculated as if crop and livestock characteristics were collected based on the sample, in order to illustrate the quality of the sample for the characteristics actually collected only for the sample. |
Annexes: 6.2.1-1 Relative standard errors |
6.3. Non-sampling error |
not applicable |
6.3.1. Coverage error |
1. Under-coverage errors |
All necessary steps are taken to ensure full coverage of the population. The Agriculture Register, finalised after FSS 2013, was further updated in April 2016 (prior to FSS2016) to add 5 273 new 'births' which had been identified as newly-active holdings on Ministry of Agriculture's administrative databases. Therefore, the Agriculture register contained 144 931 entries and was considered to be very comprehensive. The only units that could have been excluded were those farming but not registered on either of the two administrative databases (IACS & Bovine Register). However, the likelihood of a new farm not falling into one of these two databases is considered low. |
2. Over-coverage errors |
While 5 273 'births' were added to the register, it is not always easy to identify farm 'deaths'. However, page 1 of the FSS questionnaire asks the respondent to indicate if the holding has been sold or leased or if the registered holder has retired or is deceased. These units are subsequently marked as inactive and considered 'out-of-scope'. These out-of-scope units are taken into consideration when calculating survey weights, in that only in-scope responses are included when calculating the non-response weight. |
2.1 Multiple listings |
While 5 273 'births' were added to the register, it was difficult to identify farm 'deaths' and this may lead to duplicate entries on the register if the new record related to a new owner took over an existing farm holding. While every effort is made to eliminate these prior to issuing questionnaires, it was possible that some farms received two questionnaires. In some of these cases, farmers returned the second (blank) questionnaire with the completed questionnaire. To eliminate cases where two questionnaires were completed for the same holding, a thorough examination of data was carried out to identify records with identical data. This was done primarily using name and address matching, but also using several of the key variables. In all, approximately 383 duplicates were identified. These were considered inactive and out of scope and excluded from the non-response weight calculation. |
3. Misclassification errors |
Units were initially classified according to data collected in FSS 2016 which was very comprehensive, extensively validated and agreed with Eurostat. Therefore, we do not consider misclassification to have been an issue. There is the possibility that units have increased/decreased in size and economic size in 2016 and moved into a new class size as a result but this is likely to have only affected a very small number of holdings. Therefore, there was no adjustment of strata prior to weighting up. |
4. Contact errors |
The contact data was provided by the Agriculture Register. In some cases, the holder could not be reached at that address and the questionnaire was returned unopened. This occured in just 40 cases. These were considered inactive and out of scope and excluded from the non-response weight calculation. |
5. Other relevant information, if any |
Not available. |
|
6.3.1.1. Over-coverage - rate |
Over-coverage - rate |
Estimated at approximately 2.8% at the level of the sample. |
|
6.3.1.2. Common units - proportion |
[Not requested] |
6.3.2. Measurement error |
Characteristics that caused high measurement errors |
The FSS2016 Survey Instrument is a paper questionnaire issued by post and self-completed by the respondent, not by a trained interviewer. Therefore, the interpretation of certain questions is difficult to control without having a trained interviewer present during completion. However, data are validated extensively at each stage of processing for consistency (with previous responses/ external data sources) and for coherence. Re-interview/Re-surveying does not occur. |
|
6.3.3. Non response error |
1. Unit non-response: reasons, analysis and treatment |
Unit non-response occured when a sample unit declined to respond to the questionnaire, despite the issuing of three reminders. Non-response was assumed (as opposed to out-of-scope/inactivity) when a form wasn't returned. Administrative data was utilised where possible for farms which were found to be active on administrative files despite providing no response. Otherwise, imputation was used to impute certain characteristics for the non-sampled units to compile a full census. However, there was no administrative data or robust imputation method available for a small number of FSS characteristics (other gainful activities, crop rotation and manure management). Therefore, these are available for the responding units only (n=37 007) and as such are weighted variables. Non-response was taken into consideration when calculating weights for these particular variables. Full non-response was addressed by using administrative data to confirm level of activity and provide data. Therefore, bias due to non-response is considered to have been addressed. The unit non-response rate is 31.8%, non-responding units with unknown eligibility status are treated the same way as the ineligible units. |
2. Item non-response: characteristics, reasons and treatment |
As all data on bovines and crops were collected from administrative records, only variables collected in the FSS paper questionnaire were affected by item non-response. This seemed to occur mostly in the farm labour, OGA, crop rotation, manure management and training sections. The FSS is a self-completed postal questionnaire (8 pages) and as such there may be respondent fatigue by the time these sections are reached. The data being collected are complex and do not work well in a postal questionnaire with no trained interviewer present during completion. It can therefore be difficult also to determine if the cells are empty due to non-response or are in fact real zero. Where available, administrative data is used to impute for item non-response or to confirm real zero. In the absence of administrative data , data were imputed using regression if appropriate explanatory variables could be identified. |
|
6.3.3.1. Unit non-response - rate |
Unit non-response - rate |
The unit non-response rate was 31.8% at the level of the sample. |
|
6.3.3.2. Item non-response - rate |
Item non-response - rate |
This was not captured. |
|
6.3.4. Processing error |
1. Imputation methods |
Imputation techniques were used to complete the dataset for the population: Sheep: The annual Sheep & Goat Census carried out by the Ministry of Agriculture, which provides a register of all sheep producers with a reference date of December of each year. This was used to impute for missing sheep data. The number of breeding females (C_3_1_1) was taken from the Census and an expected non-breeding flock (C_3_1_99) per unit of breeding female was derived controlling for whether the farm was an upland or lowland holding (as this factor influences productivity per breeding female). Labour: Where the age of the holder was not provided or a unit was not sampled, administrative files were first checked for a date of birth. If this failed, the age at the last FSS in 2013 was checked if available and adjusted accordingly. Finally, if the age could still not be confirmed, the distribution of holder ages across all returns was examined and this distribution was used to randomly assign ages to the missing cases. In returns where the labour force section was left completely blank or in cases where the farm was not directly surveyed in 2016 , regression techniques were utilised to provide a model for labour component of farms based on all available explanatory variables including area farmed, number of livestock, age of holder, gender of holder amongst others. Time spent was also regressed on explanatory variables. Grass: Where no grassland area was provided for farms with bovines, the number of bovines in each category were used as explanatory variables in predicting a value for area of grassland. Also, imputation from administrative data or previous surveys was also used to account for unit non-response. |
2. Other sources of processing errors |
Each form was scrutinised before scanning to highlight any obvious errors. After scanning, the verification procedure ensured that any questionable cells were checked and corrected. |
3. Tools used and people/organisations authorised to make corrections |
Imputed and regression estimates were compiled by the Statisticians responsible for FSS 2016 and all work was carried out in SAS. |
|
6.3.4.1. Imputation - rate |
Imputation - rate |
Plants Harvested Green (B_1_9) and Pasture and Meadow excluding rough grazing (B_3 minus B_3_2): Total grassland was available from administrative sources, however the breakdown between Plants harvested green and Pasture and Meadow was unavailable. Therefore the proportional breakdown of pasture and meadow and plants harvested green (from questionnaire) to total grassland (from administrative data) was applied to the non-surveyed units. Imputation rate for B_1_9 and B_3=72.6% Administrative data was available for all other main characteristics. |
|
6.3.5. Model assumption error |
[Not requested] |
6.4. Seasonal adjustment |
[Not requested] |
6.5. Data revision - policy |
Data revision - policy |
Ireland does not have a revision policy. |
|
6.6. Data revision - practice |
Data revision - practice |
Some revisions may take place arising from Eurostat validation checks. Otherwise, the data is considered to be final. |
|
6.6.1. Data revision - average size |
[Not requested] |