IMPORTANT LEGAL NOTICE - The information on this site is subject todisclaimercopyright notice
Evaluation
  EUROPA > European Commission > EuropeAid > Evaluation > Methodology > Guidelines > Geo-thematic
Last updated: 19/01/2006
EuropeAid

Geographic and thematic evaluation
Guidelines for the evaluation team

EuropeAid
 

Synthesis phase (3)

 


Methodology
• Evaluation Guidelines
• Methodological bases
• Evaluation tools
• Examples
• Glossary
• Sitemap

This guideline
• Overview
• Process scheme

Detailed guidance
• Preparation
• Desk
• Field
• Synthesis

 
 

Expressing findings

The evaluation team formalises its findings on the basis of the following elements:

Findings only follow from facts, data and analysis. Unlike conclusions, they do not entail value judgments.

Findings include cause-and-effect statements related to the contribution of the support to observed changes, or the attribution of part of the observed changes to the intervention under evaluation.


Top

Confirming findings

For each question, the evaluation team submits its provisional findings to criticism in order to confirm them. A finding is considered as sound if it stands criticism such as:

  • Validity tests for statistical analysis.
  • Cross-checking with other sources of information.
  • Search for biases in the surveys.
  • Search for external factors likely to explain the detected changes even in the absence of intervention.
  • Cross-checking with findings obtained from similar research and evaluations (according to the experts involved).
  • Critical comments received from the Delegation (s) or the reference group members when preliminary findings are discussed.

When a finding entails a cause-and-effect statement, the evaluation team specifies whether it may be generalised or transferred to other contexts.


Top

Judgment and conclusions

For each question, the evaluation team formalises its responses by way of conclusions, on the basis of the following elements:

  • Evidence and findings.
  • Judgment criteria (also called reasoned assessment criteria) adopted in the first phase (desk).
  • Judgment criteria actually applied and justification for the discrepancies, if any.
  • Targets

Among its conclusions, the evaluation team identifies transferable lessons, in other words, conclusions based on generalisable and transferable findings.

Apart from the answer to each question, the evaluation team seeks to articulate all the findings and conclusions in a way that allows for an overall assessment of the intervention.


Top

Version 1 of the report

The evaluation team writes the first version of its report. This document must have the same format and contents as the final version, with the exception of the recommendations, which may be just sketched. The report consists of four parts:

  • Summary, including in particular the main findings, conclusions and recommendations.
  • Introduction presenting the assessed intervention, its logic and context, and the purpose of the evaluation.
  • Presentation of the evaluation method.
  • Detailed findings, conclusions and recommendations.

The report is limited in size (maximum 60 pages) so that it is easy to read. Details are appended in annexes.

The evaluation team leader checks that the report meets the quality criteria. He/she ensures that the report is submitted to a thorough quality control by an expert who is not part of the evaluation team. Then, the report is handed over to the evaluation manager.


Top

Presenting the report

Following are the points to be addressed:

  • Answers to questions.
  • Potential methodological limitations and judgment criteria actually applied.
  • Overall assessment of the intervention.
  • Outline of recommendations.

The evaluation team leader receives three types of comments:

  • Oral comments at the meeting.
  • Written comments after the meeting.
  • Comments from the evaluation manager on the methodological quality of the report.

Top

Version 2 of the report

The evaluation team takes the comments received into account, yet without compromising the independence of its value judgments. The process is as follows:

  • Comments dealing with methodological quality are taken into account imperatively, as long as that is possible. Whenever they are not taken into account, the evaluation team accounts for its reasons.
  • Comments dealing with the substance of the document, findings and conclusions may be taken into account or rejected by the evaluation team. The team mentions and explains the requests for dismissal in a note or annex.

The evaluation team finalises its recommendations, which are clustered and prioritised. As far as possible, alternative options are proposed, including their respective benefits and risks.

The new version of the report is handed over to the evaluation manager.


Top

Discussion seminar

When a seminar is convened at this stage, its purpose goes beyond mere dissemination. It aims at discussing the substance of the conclusions and recommendations.

  • The evaluation team updates the report's presentation slides prepared for the reference group.
  • The team leader participates in the seminar, maybe together with another member. He/she presents the report.
  • He/she takes note of the received comments, which account for the last chance to check factual data, confirm analysis and justify value judgments explicitly.
  • He/she also takes into consideration written comments received after the seminar.

The slides shown are handed over to the manager in PowerPoint version.

A meeting with the other donors may be organised on site immediately after the seminar.

Top

Finalising the report

Following the seminar, the evaluation team drafts the last version of its report and submits it to the expert in charge of quality control before it is handed over to the manager.

At this point, the evaluation team finalises the annexes and decides on presenting them under one of the following formats:

  • Printed out annexes following the report.
  • Annexes on CDROM.

The report is printed out according to the instructions provided by the terms of reference.

The evaluation team leader receives a new quality assessment from the manager. If necessary, he/she writes a note setting forth the reasons why certain requests for quality improvement have not been sustained. This response will remain attached to both the quality assessment and the report.

Examples

Final report for an evaluation:

  • at country level EN  (830 kb)
  • at regional level EN  (600 kb)
  • global sector / thematic EN  (570 kb)

Top