IMPORTANT LEGAL NOTICE - The information on this site is subject todisclaimercopyright notice
Evaluation
  EUROPA > European Commission > EuropeAid > Evaluation > Methods > How
Last updated: 08/08/2005
EuropeAid

Methodological bases
Evaluation process (How?)
Evaluation questions
Examples

EuropeAid
 

Evaluation at country level

 


Methodology
• Evaluation Guidelines
• Methodological bases
• Evaluation tools
• Examples
• Glossary
• Sitemap

Methods
• What
• When
• Why
• Who
• How

How
• Overview
• Strategy
• Questions
• References
• Design
• Data collection
• Analysis
• Judgment
• Quality
assurance

Questions
• Overview
• Selection
• Preparation
• Examples

 
 

The questions below are examples. Many are drawn from recent country evaluations undertaken by the Evaluation Unit.

Questions on relevance

  • How appropriate is the EC strategy as regards the objective of reducing poverty?
  • Have the development problems encountered by the country and their causes been sufficiently analysed and clarified to justify the choice of strategic priorities?
  • Was the strategy sufficiently concentrated and ranked?
  • To what extent has the strategy incorporated lessons learned from the experiences of the EC and other sponsors, concerning factors that facilitate or impede the success of aid in the partner country?

Top

Questions on effectiveness

Questions on sector-specific effectiveness

This site proposes a method for choosing questions specific to a particular aid sector, based on the analysis of the intervention logic. The method generates the following standard type of question:

  • To what extent has European aid in the sector contributed towards [generating effect X (specific to the sector)]?

These are questions on effectiveness (attaining a specific objective or obtaining an expected impact). Each standard question can be reformulated in many ways:

...by specifying the scope for example:

  • To what extent has European aid in the sector [and more specifically that implemented since devolution] contributed towards generating effect X?

...or by specifying the effect concerned, for example:

  • " To what extent has European aid in the sector contributed towards generating effect X [for the poorest population groups]?

...or by changing the evaluation criterion, for example:

  • Has European aid in the sector helped to generate effect X [sustainably]? (sustainability)
  • Has European aid in the sector helped to generate effect X [with a limited cost compared to ]? (efficiency or cost-effectiveness)

Top

Generic questions on effectiveness


  • To what extent have the Commission's interventions in the different sectors contributed to political dialogue focused on meeting the needs of the poorest population groups?
  • To what extent have the Commission's interventions in the different sectors helped to improve the management of public funds?
  • To what extent have the Commission's interventions in the different sectors helped to mitigate the negative impacts of economic reforms?
  • To what extent have the Commission's interventions in the different sectors contributed towards fair access to basic services of a high quality?
  • To what extent have the modalities of implementation of the strategy made it possible to meet the needs of the poorest regions as a priority?

Questions on efficiency

Questions on efficiency (also called cost-effectiveness) may be specific to an aid sector and are formulated as follows:

  • " Has European aid in the sector contributed towards generating effect X [with limited costs compared to ]?

It is also useful to ask generic questions on efficiency:

  • To what extent have the Commission's interventions made it possible to produce the expected effects with the lowest transaction costs possible?
  • Is there evidence of greater value added by Community aid in the case of actions that have mobilised the most resources?
  • In what proportion have EC resources been used to achieve the various strategic objectives?

Top

Questions on sustainability

Questions on sustainability are generally specific to an aid sector and are formulated in one of the following two ways:

  • To what extent has the Commission's intervention helped to generate [effect X] in such a way that it lasts after the end of the aid?
  • To what extent has the Commission's intervention helped to generate [effect X] in such a way that it lasts even if the aid is limited [to investments only].

Top

Questions on coherence and complementarity

  • To what extent does aid in sector X (e.g. roads) contribute directly or indirectly towards attainment of the objectives in sector Y (e.g. trade)?
  • To what extent has the design of the strategy taken into account potential interactions and conflicts with other EC policies?
  • To what extent has coordination with bilateral and multilateral interests made it possible to identify those objectives that the EC can meet better than others?
  • To what extent has the Commission contributed towards coordination between the different development partners, and to what extent has this allowed progress on the various roads to poverty reduction?


Questions on cross-cutting issues

The cross-cutting issues mentioned most frequently in recent evaluations are:

  • Gender
  • Environment
  • Good governance (decentralisation, efficient administration that is accountable and oriented towards serving the people)
  • Human rights, the rule of law, democracy, development of the role of civil society
  • Aids

Each of the following questions may concern one or more cross-cutting issues:

  • To what extent has the EC included [cross-cutting issue X] in the implementation of its interventions?
  • To what extent has the EC contributed directly or indirectly towards the development [of cross-cutting issue X]?


Questions on instruments and implementation

  • To what extent was the choice of different instruments and their combination made to generate the expected effects in the context of the partner country?
  • To what extent have the monitoring mechanisms been focused on the results and helped to improve them.
  • To what extent do the administrative procedures of allocating funds and managing actions facilitate or impede the adaptation of aid to beneficiaries needs?
  • To what extent has the implementation [of the new tool "Y' (e.g. financial support for a sector)] significantly improved the chances of obtaining the expected impacts?
  • To what extent has the design and implementation of the interventions allowed good take-up by local partners?

Top

Example of a recent evaluation

The following example presents nine questions chosen in the case of a recent country evaluation (with several important simplifications).

  • To what extent does the Commission's health strategy meet the basic needs of the population and especially of the poorest groups?
  • Is the Commission's strategy in the private sector suited to the needs of firms?
  • To what extent have the Commission's interventions concerning transport helped to improve national and regional trade?
  • To what extent have the Commission's interventions helped to ensure that investments are maintained in the transport sector?
  • To what extent have European interventions helped to reinforce the process of regional and global cooperation and economic integration?
  • To what extent have the Commission's interventions contributed towards the decentralisation process?
  • To what extent has macroeconomic support helped to improve the management of public finances?
  • To what extent does the EC ensure that there is coordination, coherence and complementarity between its interventions and the programmes of other sponsors, especially member States?
  • To what extent is the cross-cutting objective of good governance integrated into all the interventions?

Top