FAQ > mid-term evaluation (MTE)
What is the purpose of the MTE?
In accordance with Article 86 of Council Regulation (EC) 1698/2005, the MTE should assess the utilisation of resources of the European Agricultural Fund for Rural Development (EAFRD), the effectiveness and efficiency of how these resources are programmed and the results and impacts of RDP interventions against programme strategies and in achieving Community priorities. In the current programming period the MTE must be carried out in 2010.
The MTE has a distinctive and critical role as a formal mid-point review which informs about adjustments that may be appropriate for the current programmes and may also contribute to the design of subsequent programmes.
Are there guidelines on the MTE of the Rural Development Programmes?
Evaluation guidelines, including the MTE, are available in Guidance note B of the Handbook on CMEF[PDF
].
Additional, more detailed guidelines supplement the Community guidelines indicated above, Guidelines on the preparation of the MTE [ PDF
].
Read more about mid-term evaluation on this website.
What are the main steps of the MTE?
There are three main steps to be completed in carrying out the MTE:
- Preparation: activities for a good preparation of the MTE include: review of common and programme-specific evaluation questions and related indicators, data collection and information gathering, establishment of the evaluation steering group (optional but advisable), establishment of the evaluation mandate (optional but advisable) and the Terms of Reference (ToR) for contracting independent evaluator(s).
- Implementation: an independent and interactive process will lead to an effective MTE. It should include sound management and feedback procedures between Managing Authorities and independent evaluators, and a quality assessment of the MTE report. Where an evaluation steering group has been set up, it should help to guide and support the implementation process.
- Dissemination of results: evaluation is effective if findings are readily accessible and clearly understood by the target audiences.
The Guidelines on the preparation of the MTE [ PDF
] respond to these challenges, mainly concentrating on the first of these steps, however, some key considerations for the latter two steps are also described.
Read more on this website.
What is meant by review of evaluation questions and indicators?
The review of evaluation questions and indicators is part of the preparatory work for the main evaluation events. For the MTE, evaluation questions and indicators have to be reviewed to ensure an appropriate evaluation of each Rural Development Programme (RDP), capturing its specificities while also dealing with the requirements of the CMEF. In this context, both the common elements provided by the CMEF and the programme-specific (additional) evaluation questions and their related indicators must be identified. These evaluation questions and indicators should be considered and reconciled against the specificities and objectives of the RDPs, checking for ‘fit’ and precision to ensure that they are capable of comprehensively reflecting and capturing the effects of the supported interventions.
See Guidance note B of the Handbook on CMEF [PDF
].
See Guidelines on the preparation of the MTE [ PDF
].
Why set up a steering group for the MTE?
Setting up a steering group (SG) for the MTE is advisable as:
-
it accompanies and supports the MTE process
-
it involves internal and external stakeholders in the planning and implementation of the programme
-
it extends the relevance of evaluation within the implementing bodies
-
it adds value to the evaluation findings and the implementation of any recommendations or adjustments
-
it contributes significantly to the development of evaluation capacity within the stakeholder group
-
it carries out a quality assessment of the final milestone evaluation reports (suggested as a good practice).
See Guidelines on the preparation of the MTE, section 2.3 [PDF
].
See Guidance note B of the Handbook on CMEF, chapter 5 [PDF
].
What is meant by “establishment of the evaluation mandate for MTE”?
The evaluation mandate (optional but advisable) is a document that guides the preparation of the Terms of Reference (ToR) for the mid-term evaluator and the work programme of the steering group (SG). It specifies the context, scope, timing, and objectives of the MTE.
The preparation of the evaluation mandate is mostly initiated by the Managing Authority (for example, the unit responsible for evaluation), or by the steering group (if applicable).
The evaluation mandate should be approved by the Managing Authority confirming its consistency with the requirements of the CMEF. Good practice would also suggest approval by the steering group.
See Guidelines on the preparation of the MTE, section 2.3 [PDF
].
What is required in establishing the Terms of Reference for MTE?
The Terms of Reference (ToR) for the MTE is prepared by the Managing Authority and this sets the framework for all activities to be carried out by the external evaluator for the MTE process. It is therefore important that the ToR is complete and clear, and includes the following topics:
(a) The context of the MTE
(b) Scope of the MTE
(c) Objectives of the evaluation
(d) Evaluation questions
(e) Tasks to be performed by the evaluator
(f) Timing and content of deliverables
(g) Organisation of the work
(h) Sources and documentation
See Guidelines on the preparation of the MTE, section 2.5 [PDF
].
Are there guidelines available about the Terms of Reference for the MTE?
General information on establishing the Terms of Reference (ToR) for evaluation activities is available in Guidance note B of the Handbook on CMEF [PDF
].
Further explanations concerning technical specifications on ToR are outlined in Annex 1 of the Guidelines on the preparation of the MTE [PDF
].
What expectations does the European Commission have for the mid-term evaluations of RDPs which have to be submitted by 31 December 2010 considering that the implementation of many of the programmes is at an early stage?
Article 86 (6) of Council Regulation 1698/2005 specifies what the European Commission expects from the mid term evaluation:
"… shall examine the degree of utilisation of resources, the effectiveness and efficiency of the programming of the EAFRD, its socio-economic impact and its impact on the Community priorities. They shall cover the goals of the programme and aim to draw lessons concerning rural development policy. … identify the factors which contributed to the success or failure of the programmes’ implementation, including as regards sustainability, and identify best practice".
The European Commission expects these provisions to be met to the greatest possible extent. Addressing the common measure specific and horizontal evaluation questions needs to be considered in the MTE as stated in the Handbook on CMEF (Guidance note B – Evaluation Guidelines). Preliminary impacts should be assessed, however, if this is not yet possible, future impacts could be inferred from the analysis of the results obtained so far. Trends should be analysed and described including (where applicable) those flowing through from previous programme periods. The MTE should also propose measures to improve the quality of RDPs and their implementation with a view to informing possible revisions of the programmes. The MTE has also to prepare for the ex post evaluation and, more generally, for future ongoing evaluation activities.
Can data from previous periods be used for the MTE and which period should be covered in the MTE?
The reference period covered by the MTE for 2007-2013 should as a minimum cover the programming years 2007-2009 (inclusive) but could extend into 2010, depending on data availability and therefore may vary between programmes. In the MTE report, the reference period should be clearly described and justified.
Additionally, where appropriate a link to the previous programming period should be established as some measures are carried over from the previous RDP and previous programme measures may also be generating effects and contributing to objectives of the current programming period. These have to be assessed in the context of the MTE. Therefore, the assessment can be based also on the experience of previous programming periods.
Should the Health Check and the Economic Recovery Plan be considered in the MTE?
Very little data about these programme modifications is expected to become available before June 2010, when the Member States must submit the new indicators to the Commission. Therefore, for most programmes, the assessment of impacts related to the Health Check and the Economic Recovery Plan is likely to provide only limited information. However, it should be reported on how
Is there information available about tendering procedures for the MTE in other countries?
In July 2009, the Evaluation Helpdesk published the results of a survey which was carried out on the state of preparation of the Member States for the mid-term evaluation (MTE). This document provides information about the tendering procedures used for the MTE and ongoing evaluation in the Rural Development Programmes of the Member States that participated in the survey.
Download the survey [PDF
].
Should all common evaluation questions (measure specific and horizontal) be dealt with in the MTE?
Addressing the common measure specific and horizontal evaluation questions needs to be considered in the mid-term evaluation (MTE) as stated in the Handbook on CMEF (Guidance note B – Evaluation Guidelines [PDF
]). Due to the relatively short period of programme implementation up to the MTE stage, the evaluation results could be regarded as intermediate results, with a more comprehensive picture of the Rural Development Programmes’ effects emerging from the ex post evaluation. However, answering the evaluation questions is an important contributor to decisions about possible revisions of the programmes according to the evaluation results.
Do the National Rural Networks need to be assessed in the mid term evaluation and if yes, to what extent?
Most of the National Rural Networks (NRNs) are funded as part of the technical assistance element of Rural Development Programmes. Four of the Member States with regional programmes (
The mid term evaluation (MTE) should therefore assess, to what extent the objectives of the networks have been achieved at this stage in the programming cycle, as well as how the networks contributed to the realization of the overall programme objectives. As part of the programme, a number of horizontal evaluation questions may also apply to the networks (See Guidance note B of the Handbook on CMEF, p. 31 [PDF
]).
Do all the common evaluation questions have to be answered in the evaluation reports?
All the common evaluations questions that relate to measures included in a given rural development programme have to be addressed in the evaluation reports.
Do the horizontal evaluation questions apply to all axes, for example do the evaluators look for environmental effects and employment effects of all measures applied?
The horizontal questions shall assess the contribution of the whole programme to the achievement of the overarching objectives. Therefore the horizontal questions apply to all axes and the evaluators have to look for effects of all measures applied including negative effects.