IMPORTANT LEGAL NOTICE - The information on this site is subject todisclaimercopyright notice
Evaluation
  EUROPA > European Commission > EuropeAid > Evaluation > Methodology > Evaluation tools
Last updated: 15/12/2005
EuropeAid

Evaluation tools

EuropeAid
 

Which combination of tools?

 


Methodology
• Evaluation Guidelines
• Methodological bases
• Evaluation tools
• Examples
• Glossary
• Sitemap

Evaluation tools
• What type of tools?
• When to use which
tools?

• Which combination
of tools?

• What specific
constraints and
requirements?

• Check list for the
tool's implementation

 
  The combination rationale

Some tools require the implementation of other tools prior to their use. Such is the case when a tool yields useful information for the implementation of another tool, or when bringing a different viewpoint to the analysis, it strengthnens or nuances the conclusions reached which another tool. Before taking the decision to implement complex tools (such as judgement tools), the evaluator should check whether preliminary information which is capable of improving their performance are available, and find the optimum tool capable of yielding such information.
The most frequent cases of combinations of tool are listed in the table below. For example, the first line shows that objectives diagrams and impact diagrams require or may require the implementation of interviews and focus groups.

Tools required by other tools

Additional tools used



Tools implemented
Objectives
diagram and
impact
diagram
                       
Problem
diagram
                       
Decision
diagram
                       
Interview                        
Focus group                        
Survey                        
Case study                        
SWOT                        
Indicateurs                        
Expert panel                        
Cultural and
social
analysis
                       
Multicriteria
analysis
                       
Cost-
efectiveness
analysis
                       

The table is indicative and other combinations can be developed in particular contexts. Usually, collection tools (interviews, focus group and surveys) are the most frequently combined with analysis and judgement tools, because the latter require specific information for their implementation. Thus, special care should be given to analysis and judgement tools, so as to ensure the homogeneity of the methodology and the maximum performance of the tools throughout the evaluation.


Top

  The tools' frequency of use

The table shows that the interview is the tool the most frequently used by other tools, including collection tools (which is not surprising). Conversely, case studies are seldom used and belong to the category of meta-tools requiring the support of all the collection tools and, if needs be, analysis tools.
Although presenting an exhaustive table of the tool's combinations is not possible, recommendations should be made to ensure the homogeneity of the tools performance all through the evaluation:

  • Some tools are frequently used by other tools. For example, the interview is the collection tool the most often used by other evaluation tools.
  • Most of the tools are multifunctional. The decision to use a specific tool should thus be based on its conditions of use and on the constraints related to the implementation of additional tools.
  • Because of the polyvalent nature of the majority of the tools, criteria for rationalisation and optimisation must support their selection. For example, evaluation questions should be classified depending on the tools selected and vice-versa.

Example of combination

Testing of two country evaluation tools - the survey and the focus group - during the evaluation in Benin.
The test checked the possibility of providing elements to answer the following evaluation question through the implementation of a combination of two tools: "To what extent are the European Commission's sanitary strategy and interventions adapted to the fundamental needs of the population, and particularly the poor?"
The objectives of the test were to collect and analyse information and points of view from beneficiaries, in order to assess the global trends of the service provided by health centres. The test combined three surveys and four focus groups.
Three surveys:

  • Two structured questionnaires, one addressed to households and the other to the users of the health centres, which resulted in a total of 660 respondents
  • One open-ended questionnaire addressed to the local authorities, health professionals and people in charge of insurance companies and local associations, which resulted in a total of 100 respondents.

Four focus group investigations:

  • Two focus groups of beneficiaries (countrywomen) in two different villages.
  • One focus group with midwives and nurses.
  • One focus group with doctors from the private and public sector.

The last 2 focus group investigations were set up to complement the outcomes of the 2 focus groups with beneficiaries and that of the surveys. The goal was to confront different perspectives on a same situation where a few dysfunctional elements could be raised.
The confrontation of the survey's outcomes with that of the focus group contradicted the hypothesis at the start of the reorganisation of the public health system implemented by the European Commission in Benin. It also asserted that the access to health care in the public sector was very limited for the poor population.

Top