6. Accuracy and reliability |
Top |
|
- |
6.1. Accuracy - overall |
Main sources of error |
Various types of errors and warnings may affect the quality of the results obtained from the FSS. One such error that affects the overall quality of the survey is the observation (or collection) error, which should be corrected wherever possible. Other errors arise during input, encryption or data processing. Main sources of error are sampling errors, over-coverage, non-response and measurement errors. |
|
6.2. Sampling error |
Method used for estimation of relative standard errors (RSEs) |
The estimations in the Farm Structure Survey 2016 are generally in the form of totals and ratios. The domain of estimating is the country level (Republic of North Macedonia) and the eight regions. In the estimation procedure data on individual agricultural holdings are weighted with design weights adjusted for non-response. The deviations of estimated data from the sample and hypothetically true data on the population are calculated as standard deviations and coefficients of variation (relative standard errors). All calculations are made with the SAS 9.1 statistical software package. SAS PROC SURVEYMEANS procedure was used for the calculation of standard errors and coefficients of variation. The sampling design was considered during the calculation of RSEs. |
|
6.2.1. Sampling error - indicators |
1. Relative standard errors (RSEs) - in annex
2. Reasons for possible cases where precision requirements are applicable and estimated RSEs are above the thresholds |
Almost all precision requirements stipulated in Annex IV "Precision Requirements" of the Regulation 1166/2008 have been met except for bovine animals and sheep. |
Annexes: 6.2.1-2. Relative standard error_FSS 2016 |
6.3. Non-sampling error |
- |
6.3.1. Coverage error |
1. Under-coverage errors |
During the FSS 2016 some changes in the farms were obtained. In most cases changes in the farms were related to the changes of the farm holder. During the survey, 2323 were newly-established agricultural holdings. Such holdings were not recorded in the Farm Register, but they were surveyed as they belong to the target population. All new agricultural holdings were added into the Statistical Farm Register. |
2. Over-coverage errors |
We updated the Statistical Register of Agricultural Holdings (we excluded ineligible family farms from the frame, which turned out that they didn’t belong to the target population). Between the agricultural censuses 2007 and 2016 the farm register was updated partially based on the information of FSS (2010, 2013) and regular annual sample surveys. Weighting factors were calculated on the basis of eligibility status of agricultural holdings, with the formula (responses + nonresponses)/responses - on the level of strata. |
2.1 Multiple listings |
There were no multiple units in the frame. |
3. Misclassification errors |
Urban, rural areas and regions as stratification variables are the same in the sample and in the data processing and there are no differences in the classifications and no impact on weights. |
4. Contact errors |
For individual agricultural holdings the interviewers had a list of holders (in the territory for which they were engaged). This list contained main contact data for the holder. As they interviewed door-to-door, they checked the contact data on the list, and recorded the changes (if any). For legal entities, the contact data were checked with other sources in SSO and if it was needed they were additionally contacted by phone or e-mail. |
5. Other relevant information, if any |
Not available. |
|
6.3.1.1. Over-coverage - rate |
Over-coverage - rate |
From the initial list of agricultural holdings 2902 of units with ceased activity were surveyed as they belong to the target population and it turned out that they didn’t belong to the target population. The over-coverage rate was 12.6%. Some of the units have changed their owner, others have ceased their farming activities. For the aggregates, the coefficients of the other eligible holdings have not been changed following the ineligibility of these units. No weight correction due to over-coverage has been done. |
|
6.3.1.2. Common units - proportion |
[Not requested] |
6.3.2. Measurement error |
Characteristics that caused high measurement errors |
Many sources, which occurred in the period of data collection, had influence on measurement errors: 1. questionnaire, 2. interviewers, 3. respondents, 4. data collection.
1. Questionnaires The data in the survey were collected with 2 types of questionnaires designed on the basis of the EC Regulation 1166/2008.
2. Interviewers Having well-prepared interviewers is an essential part of collecting complete and reliable data. For this survey, the SSO engaged interviewers to carry out the FSS. These interviewers were trained and made familiar with the survey to ensure that the data are collected correctly. The training was done by persons responsible for methodology from the SSO, for all responsible persons (supervisors) for the survey from each of the 8 regional statistical offices. After that, the responsible persons from the regional statistical offices trained the interviewers. The training programme included methodology, questionnaires, classifications, method of interview.
3. Respondents Target group of the survey are agricultural holdings on the territory of the Republic of North Macedonia. Proxy interviews are also used in agricultural households because most of the household members during the season periods are working in the fields and they are not at their homes.
4. Data collection The mode of data collection in FSS is via the PAPI method.
Data collection, coding and control of the collected data were organised in the eight regional offices. A number of enumeration districts were given to each interviewer. For each enumeration district, they got a list of a number of households. |
|
6.3.3. Non response error |
1. Unit non-response: reasons, analysis and treatment |
For individual agricultural holdings, the interviewers were trained to give their maximum to obtain unit response. If it was not possible, the special set of reasons for unit non-response were given in the questionnaire. The main reasons for non-response were the following: holders consider themselves as “non-agricultural holdings” and general refusal because they didn't like to participate in the survey. They were adequately treated in weighting procedure. Weighting factors were calculated with the formula (responses+nonresponses)/responses - on the level of strata. Legal entities with unit non-response were additionally contacted by phone. |
2. Item non-response: characteristics, reasons and treatment |
For individual agricultural holdings, the item non-response was negligible as during the face-to-face interviews the appropriate tables of the questionnaires have been completed by the enumerator. For legal entities for some important item non-responses, the legal entities were additionally interviewed by phone. |
|
6.3.3.1. Unit non-response - rate |
Unit non-response - rate |
If the non-response rate is considered as the share of the non-respondents among all eligible agricultural farms then the unit non-response rate is 17.2%. |
|
6.3.3.2. Item non-response - rate |
Item non-response - rate |
Item non-response was corrected during the data entering phase by the supervisors and their knowledge. In this case it was not possible to count them. |
|
6.3.4. Processing error |
1. Imputation methods |
Not applied. |
2. Other sources of processing errors |
Data processing was carried out in two phases: 1/ Data-entry phase A large number of edit checks between questions in questionnaires were implemented for ensuring data correctness and consistency. During the data entry phase, data entry operators were prevented from entering some outliers and some impossible modality answers without on-line warnings. 2/ Data processing phase After the data-entry phase, further data checking and editing was performed by the subject-matter department. Initially, data were checked whether all questionnaires have been entered and completed. Data checking was done by the national checks (prepared by the subject-matter department) and Eurostat’s programme for validations with warnings and errors. All these validations were checked one by one and all errors were corrected. Also, all warnings were checked and if there were no mistakes they were approved. |
3. Tools used and people/organisations authorised to make corrections |
Supervisor in SSO |
|
6.3.4.1. Imputation - rate |
Imputation - rate |
Not available. Item non-response was corrected during the data entering phase by the supervisors and their knowledge. In this case it was not possible to count them. |
|
6.3.5. Model assumption error |
[Not requested] |
6.4. Seasonal adjustment |
[Not requested] |
6.5. Data revision - policy |
|
6.6. Data revision - practice |
Data revision - practice |
Preliminary data for main indicators from Farm Structure Survey were published in News Release in January 2017. Publication with data from Farm Structure Survey and Farm Typology were published in December 2017. |
|
6.6.1. Data revision - average size |
[Not requested] |