IMPORTANT LEGAL NOTICE - The information on this site is subject todisclaimercopyright notice
Evaluation
  EUROPA > European Commission > EuropeAid > Evaluation > Methodology > Evaluation tools > Decision diagram
Last updated: 30/10/2005
EuropeAid

Decision diagram

EuropeAid
 

How should the decision diagram be used?

 


Methodology
• Evaluation Guidelines
• Methodological bases
• Evaluation tools
• Examples
• Glossary
• Sitemap

Decision diagram
• definition
• why and when
• construction stages
• how
• quality control
• examples
• bibliography

 
 
HOW MANY DIAGRAMS SHOULD BE ESTABLISHED?

Diagram of strategic decisions

Usually, the number of the diagrams to be constructed should correspond to the number of the strategy papers drafted during the evaluation period.

Evaluators and evaluation managers may decide to work on the most recent document only, when required. Indeed, some of the early documentation needed for the establishment of the decision diagram may be too old to guarantee its availability and that of its authors.

Whatever the case, it is recommended that a decision diagram for each possible medium-term strategic revision is prepared.


Diagram of operational decisions

Such a diagram is usually prepared for each development assistance programme. As strategy papers may cover several programmes, the evaluators may have to study 4 or 5 programmes for an evaluation in a twelve-years timeframe.

The construction of such a large number of diagrams represents a significant amount of work and cost, and yields uncertain results, being dependant on the availability of the information (documents and informants). Thus, the evaluators and evaluation managers may agree to focus on programmes relating to the most recent strategy paper.

To find out more:

Top

  HOW SHOULD THE DECISION DIAGRAM BE USED?

Recommendations for country evaluations

The general recommendations about the use of decision diagrams are derived from the experience drawn from their use in the particular context of country evaluations.

Decision diagrams should be established whenever the evaluation studies the relevance of the objectives with regard to the European Union's goals, the country specificities and the overall context.

Previous evaluations show that, excluding extreme situations, objectives and interventions are usually relatively coherent, although the appropriateness of the choices made is not always indisputable. The decision diagram is likely to alleviate this particular limitation.

The use of decision diagrams in thematic and sector-based evaluations appears possible, but should be tested, and adapted where necessary.


Evaluation of the relevance of the strategy and planning

The explanations and justifications for the choices stated in the documentation and from informants should be evaluated.

Thanks to their experience or with the support of external experts, evaluators should formulate their own views about the relevance of the explanations which are provided for each decision-making choice. They should be able to judge to what extent the rejection of an objective seems justified.

The selection of the overall objective (following the rejection of other possible overall objectives) should be consistent with the strategic goals formulated by the European Union's institutions (Council, Parliament, Commission). The justification of the subordinate objectives is based on coherence with the overall objective or the ones resulting from it, and with the decision chain's inputs.


Evaluation of the quality of the partnership

The partnership between the country's authorities and the representatives of the Member States can be evaluated through the study of specific input flows.

Thus, a judgement can be formulated about the importance given to the following topics during the strategy and planning decision process:

  • Member States' strategies, programmes and agreements
  • Member States' opinions about the European Union's interventions
  • Priorities and agreements of the partner government
  • Partner government's wishes about the European Union's interventions
Evaluation of the 3Cs

The following can also be analysed:

  • The coherence of the development assistance strategy with other European policies (migration, trade, agriculture, etc.)
  • The coordination between the co-operation policies of the Commission and Member States
  • The complementarity of the Commission's programme planning with that of other donors (including the EIB)
Additional tools

These tools can be used when the decision-making factors are not explicit in the documentation collected, or when their interpretation is uncertain.

They are especially useful for evaluations where the terms of reference explicitly include an analysis of the decision-making process (thematic evaluations about the decision-making process).

The evaluators can present their successive judgements in a series of tables completing the diagram.

Validation of the diagram by the decision-makers consulted during the test

Table C: Evaluation proposed for the selection of objectives
Code for the decision-making Decisions made according to the final diagram Justification based on the final diagram Observations by the decision-makers consulted Judgement of the evaluators
         
         
         

In Table C, the columns gather data concerning:

  • The decision-making point which is identified in the sequence of the decision-making (the code is the same as the one chosen in the first column of Table A (stage 4))
  • Decisions made according to the final diagram: determination of the selected and rejected objectives
  • Justification based on the final diagram: includes the findings of the test (reported in the fourth column of Table B (stage 5))
  • Observations by the decision-makers consulted: approval (with or without reservation) or rejection of the decisions and their justification
  • The judgement of the evaluators: agreement (with or without reservation) or rejection of the observations made by the decision-makers

Validation of the diagram by an expert panel including the Commission's decision-makers and independent experts
The evaluation may be subjected to a panel composed of representatives of the Commission's services (geographic departments, delegations), and independent and recognised experts. The findings could be reported in the following table:

Table D: Final evaluation of the selection of objectives
Code for the decision-making Decisions made according to the final diagram Opinions in favour of the final diagram Opinions against the final diagram Judgment of the evaluators
         
         
         

In Table D, the columns gather data concerning:

  • The decision-making point, which is identified in the sequence of the decision-making (the code is the same one chosen the first column of Table A (stage 4))
  • Decisions made according to the final diagram: determination of the selected and rejected objectives (second column of Table C)
  • Opinions in favour of the final diagram: opinions of panel members relating to the final diagram
  • Opinions against the final diagram: opinions of panel members relating to the final diagram
  • The judgment of the evaluators: agreement (with or without reservation) or rejection of the observations made by the decision-makers

To find out more :

Top

  HOW ARE THE FINDINGS PRESENTED?

Intermediary documentation

The intermediary documentation is addressed to the managers of the evaluation and the steering committee. It must include all the graphs and tables in detail.


Final report

The final report is addressed to a wider public, more interested in the findings than the evaluation's methodology.

It is recommended that analytical graphs and tables should be placed in an annex and a synthesis of the findings should be shown in the main report under three headings:

  • Relevance of the strategic and planning decisions
  • Description of the partnership between Member States and the beneficiary government
  • Coherence, Co-ordination and Complementarity (3 Cs)

A simplified diagram of the strategic and operational decisions may usefully be included in the main report.


Presentation of the final study

It may be useful for the evaluators to use the standard decision diagram during the presentation of the evaluation's findings.

To find out more:

Top

  WHAT ARE THE PRECONDITIONS FOR THE ESTABLISHMENT OF A DECISION DIAGRAM?

Human resources and working arrangements

Type of work required for the design of a decision diagram
Tasks Types of abilities Categories of professionals Number of working days (estimation)
Identification

Collection of the documentation

Examination of the documentation

Construction of the diagrams
Knowledge of the European Commission's strategies and development programme procedures

Knowledge of the documentation resulting from the procedures

Fair knowledge of computer tools
Junior or medium professionals 5-10 days
Analysis of the decisions identified during the examination of documentation

Reconstruction of the diagrams

Test of the diagrams

Establishment of the final diagrams
Logical process of thinking

Experience in the fields covered by the strategies and programmes

Specific knowledge of the country, sector or theme under study
Multidisciplinary team of experienced evaluators, whose specialities should cover the thematic scope of the evaluation 5-10 days
Freelance experts in specific fields 5-10 days

Travelling expenses

Strategy papers used to be prepared under the responsibility of the Commission's Directorates-General (Relex or Dev) which are also responsible for the planning stage.

EuropeAid is in charge of the project design. Thus, the majority of the useful documentation (the whole baseline documentation) can be found at the Commission's Headquarters in Brussels. The delegations have taken part in the drafting process of the most recent documents and their responsibilities in this area will grow.

In addition, decision-makers from ACP countries benefiting from European assistance may in principle take part in the drafting process of the strategies and programmes which have been co-signed.


Computer devices

Specialist software can support decision-making. Yet, these devices do not seem relevant to the evaluation context, where spreadsheet programs or PowerPoint should be sufficient.


Top