LEADER Monitoring and Evaluation
6. Data and Information
Why is advance planning of this so important?
Ensuring good availability of data and information in the right form and at the right time is one of the most important contributors to an effective, robust and efficient approach to evaluation. On the other hand the failure to plan how data and information will be provided for evaluations is one of the greatest frustrations for all involved. The importance of this cannot be overstressed, if the data are not there the indicators cannot be used, the judgement criteria1 cannot be applied, and the evaluation questions cannot be answered. Getting it right first time is so much easier for all concerned and this must be the priority for everyone.
If experience shows how important this was in the past it is even more important for LEADER now with the need to ensure the fit between Local Development Strategy (LDS) monitoring and evaluation and the Rural Development Programme (RDP)2.
LAGs will be required to:
- contribute to the evaluation of the RDP as a whole;
- contribute to the assessment of the added value of the approach; and
- undertake the monitoring and evaluation of their own LDS.
The monitoring and data collection arrangements necessary in order to capture the added value of LEADER and its contribution to the RDP are complex and need careful planning.
What are the possible problems and pitfalls we need to avoid?
For evaluators poor data availability causes delays due to having to seek out sources or wait for information to be delivered. When it arrives it may be in the wrong format, inconsistent units or the wrong time sequence. Worse still it may not be available at all making effective evaluation impossible or much more expensive, requiring different tools or less suitable methods.
One common effect is to force evaluators to rely on surveys rather than monitoring or reporting data, given the variation this introduces and the frequent reliance on recollection this is a much less reliable and robust approach. This is most particularly so when evaluating qualitative effects where sound baselines are essential in measuring change.
For LAGs and Managing Authorities (MA) poor data availability also can cause delays and may result in additional tasks or workload demands in sourcing information. This may involve working from application forms rather than data bases or even in going back to beneficiaries. Consequently other work can be displaced and the quality of evaluation outputs may be reduced. It can also result in multiple requests for clarifications.
Requests to beneficiaries to provide data and information retrospectively can meet with difficulties or even resistance. At the very least this involves them in going back to records (if available), low response rates are normal and data quality can be questionable. This is even more problematic for qualitative aspects.
This can all contribute to evaluation being perceived as a burden. Given the importance of reducing such burdens to the LEADER method avoiding such additional tasks is clearly a priority.
What should we do?
Essentially the solutions are very simple. LAGs and MAs should develop and implement plans to ensure the availability of the right data in a consistent form at the right time, and this applies equally for self evaluation and for external evaluations. This means that data requirements need to be agreed in advance, anticipated and specified and a systematic approach put in place from the outset to make sure these needs are met. DG AGRI provides a set of minimum specified data items which is required to be monitored and reported. The collection and sharing of data should be planned accordingly from the LAG level. Monitoring and reporting according to predefined targets may require some sort of shared database tools. Evaluation plans at both RDP and LDS level should clarify the different responsibilities and tasks related to the provision of data needed for MA and LAGs’ monitoring and evaluation activities. At MA level this will be included in the RDP.
LAGs will need to consider both the MA requirements and what they will define and need themselves to effectively monitor and evaluate their own LDS and LAG. Using the evaluation timeline as a basis this involves retro planning to meet future needs, putting the system in place and matching this to the indicator framework and any evaluation questions the LAG has decided and wishes to consider.
What does this involve in practical terms?
The priority here is mapping what is available, what is needed (including who needs what information and data and when) and the possible sources. This involves specifying the data requirements at the right time to meet the evaluation and evaluators needs. Once you know what you need, the sources and any actual or potential gaps this should allow you to identify any potential bottlenecks and/or potential issues related to obtaining the necessary data e.g. data protection issues or access to suitably disaggregated data. This then allows you to identify actual or potential solutions or take steps to develop solutions and allocate responsibilities.
The main thing LAGs then need to do is plan the data and information collection processes and sources. This means designing these requirements in to application and claim forms and into the LAG monitoring and reporting databases, this may involve supplementing core forms or tools provided e.g. by the MA.
Informing, involving and empowering those who will be working with the data so that they understand its importance can help improve accuracy and reliability. Often data entry is a clerical or administrative task, if people don’t understand why what they are doing is important or have any basis to judge the realism of the numbers they are working with errors can creep in.
An important factor in this is the strengthened requirement on beneficiaries to provide information and contribute to monitoring and evaluation3. For the 2014 – 2020 period each RDP is required to record all supported projects in an Operations Database including data items fed in directly by application forms. There will also be specific data items required for LEADER. This should also help to improve data availability, quality and accessibility.
1 Judgement criteria are specific aspects of the intervention being evaluated which allow its merits or success to be assessed; they are derived from evaluation questions and associated with specific indicators.
2 Helpdesk of the European Evaluation Networks for Rural Development, see LEADER section in Guidelines: Establishing and implementing the evaluation plan of 2014-2020 RDPs
3 Regulation (EU) No 1305/2013 Art 71 [PDF 
]