FAQs > Indicators

What is an indicator?

An indicator is a tool to measure the achievement of: an objective; or a resource mobilised; or an output accomplished; or an effect obtained; or a context variable (economic, social or environmental).

What is a SMART indicator?

An indicator should be specific, measurable, available/achievable in a cost effective way, relevant for the programme, and available in a timely manner (SMART).

Do indicators always have numerical values?

Generally indicators have numerical values. However, in cases where indicators cannot be measured with quantitative (statistical) data, qualitative assessments or logical assumptions may be used.

What types of indicators are used in the CMEF?

There are 5 types of indicators referred to in the CMEF:

• Baseline indicators can relate to the general socio-economic context of the programme area (context-related baseline indicators) and to the state of the economic, social or environmental situation in direct relation with the wider objectives of the programme (objectives-related baseline indicators);
• Financial (input) indicators refer to the budget and other resources allocated to the programme;
• Output indicators measure the activities directly realized within the programme;
• Result indicators measure the direct and immediate effects of the intervention and provide information on eventual changes that have taken place;
• Impact indicators refer to the benefits of the programme both at the level of the intervention but also more generally in the programme area. They are linked to the wider objectives of the programme.

In Annex 1, Guidance note A (Choice and use of indicators) of the Handbook on CMEF, the different types and the use of indicators are explained. In Annex 3 (Indicator guidance) Guidance notes F - K of the Handbook on CMEF the indicator fiches for all types of indicators can be found.

You can download all documents from:

https://ec.europa.eu/agriculture/rurdev/eval/index_en.htm

What are indicator fiches? How are they used?

Indicator fiches are summary descriptive tools providing information about specific indicators including: what indicators (type and definition), why you should use them (link with the measures) and how you should measure them (unit and data source).

Fiches for all types of indicators can be found in Annex 3 (Indicator guidance), Guidance notes F – K of the Handbook on CMEF.

What are common and additional indicators?

The CMEF establishes a list of common indicators for outputs, results, impacts and baselines. These common indicators are presented in the measure fiches for the given Rural Development programmes.

Since common indicators may not fully reflect all effects of programme activities, it is necessary to complement the common indicator set by defining additional indicators to capture the full range of intended effects of a given programme, in particular for national priorities and site-specific measures.

Additional indicators are sometimes called programme specific indicators. Examples of additional indicators are presented in Guidance note K of the Handbook on CMEF.

See Guidance note A of the Handbook on CMEF (Choice and use of indicators).

See Guidance note K of the Handbook on CMEF (Examples of additional indicators).

Is it obligatory to establish additional (programme specific) indicators?

Yes. The Handbook on CMEF (Guidance note A) states: “Since common indicators may not fully capture all effects of programme activity, in particular for national priorities and site-specific measures, it is necessary to define additional indicators within the programmes. Such additional indicators should be developed by Member States and programme partnerships in a flexible manner, but in accordance with the general principles governing the use of indicators in the CMEF.”

Guidance on the use of additional indicators and examples of additional indicators are provided in Guidance notes A & K of the Handbook on CMEF.

Should fiches be developed for additional indicators?

It is recommended to provide an indicator fiche for each additional indicator to facilitate their use in monitoring and evaluation (Guidance note A of the Handbook on CMEF). It gives the opportunity to check whether:

• the type and use of the indicator are well defined
• the indicator is relevant
• the indicator meets accepted quality criteria
• the indicator is developed in accordance with the general principles governing the use of indicators in the CMEF

See Guidance note A of the Handbook on CMEF.

What are alternative indicators?

Guidance note A of the Handbook on CMEF stipulates “Where data required for a common indicator is not available in a Member State at national or regional level, alternative national or regional indicators should be compiled with appropriate data. Where no data is available at EU or national level, a qualitative assessment is acceptable.”

However, replacement of a common indicator by an alternative indicator or a qualitative assessment should be avoided and should be thoroughly justified. It is recommended to maintain the common indicator, and establish an additional indicator in cases where it would provide more complete and better information.

See Guidance notes A of the Handbook on CMEF.

What are the implications if a Member State cannot follow the methodology of the Handbook on CMEF for a certain common indicator?

Article 80 of the Council Regulation (EC) No 1698/2005 provides for a limited number of common indicators applicable to each rural development programme. Member States are responsible for organizing the production and gathering of the data required for evaluation purposes (art. 84(5) of the Council Regulation 1698/2005).

Guidance note A of the Handbook on CMEF stipulates “Where data required for a common indicator is not available in a Member State at national or regional level, alternative national or regional indicators should be compiled with appropriate data. Where no data is available at EU or national level, a qualitative assessment is acceptable.”

In either case, as provided in the indicative outline of the evaluation reports (see Guidance note B of the Handbook on CMEF, point 7), evaluators should:

1. explain the methods they use,
2. describe the data sources and how the indicators are calculated, and
3. reveal relevant problems or limitations.

See Guidance notes A, B of the Handbook on CMEF.

See also FAQ About Indicators: What are alternative indicators?

What is the role of indicators for the assessment of programme impact?

In general terms, indicators have to be considered as methodological tools supporting the assessment of an intervention in terms of its output, results and impacts.

The programme's impact should be assessed against their objectives, and related target levels. The common impact indicators need to be complemented by additional indicators specific to the programmes which take into account the full range of objectives and sub-objectives at programme level, as well as national priorities.

Is it necessary to quantify all impact indicators?

Article 85 of Council Regulation (EC) 1698/2005 refers to "quantified targets, particularly in terms of impact in relation to the baseline situation". In line with this, evaluators have to make an effort to provide quantitative assessments of the impact indicators to the extent possible.

If quantification of impact indicators is not possible, the evaluators should provide estimations or qualitative assessments.