Decision diagram

What are the decision diagram's construction stages?

• Stage 1: determination of the points when the decisions were made
• General principle
• Determination of the period allocated for the selection of objectives
• Stage 2: drafting of questions
• Identification of the rejected objectives
• Wording of questions
• Stage 3: collection of useful information
• What is useful information?
• What specific information is required for country evaluations?
• How is useful information collected?
• Is the information collected sufficient?
• Stage 4: constructing the temporary diagram
• Preliminary analysis of the information
• Draft of the explanations
• Construction of a temporary diagram
• Stage 5: testing of the temporary diagram
• Stage 6: constructing the final diagram

Once the preconditions are fulfilled, the drafting process of the decision diagram continues in two steps:

• Determination of the points at which the decision-making took place (selection of the objectives)
• Identifying, collecting and analysing of the relevant information

The process cannot start with the collection of documentation because useful information cannot be found in one or more identified documents. It has to begin with the definition of the scope of such a collection.

The determination of the objectives at various levels (overall, intermediate and operational) implies a series of choices which can be translated into a question: why such an objective and not another one?

The question's wording therefore orients the collection of the information judged to bring answers. The relevance of these answers is checked, and may result in seeking new information (written or verbal), and even rewording some of the questions, particularly those concerning the objectives which are rejected.

The decision diagram (existing or to be constructed) identifies the points at which the implicit or explicit selection of the objectives were made.

Whilst the objectives diagram assesses the internal coherence of the objectives system, the decision diagram illustrates the external factors corresponding to each choice in the decision-making and resulting in the selection of one objective over another.

Usually, the establishment of the decision diagram follows the construction of the objectives diagram. This sequence should be respected.

The objectives diagram is the basic tool for identifying the points at which the decision-making takes place. Apart from the definition of the overall objective, each intersection of the diagram represents a decision-making point. For example, in the objectives diagram, 3 intersections are illustrated, which stimulates the following questions:

Why have these objectives been selected? Why have others been rejected?

The decision diagram questions the choices concerning:

• The initial selection of the overall objective
• The subsequent selection of the intermediate and operational objectives

The planning process should be both strategic (implementation of the best means to achieve objectives of different levels) and empirical (selection of the means likely to achieve specific objectives). Whatever the planning process, the choices will be made in accordance with the objective diagram.

Each of the decision-making points should be given an identification code (which may include a date), to facilitate its transfer to analytical tables (see stage 3).

The questions focus on the justification of the selection of objectives. The identification of the rejected objectives helps the accuracy of the wording of questions. How are these objectives identified? At each decision-making point, the evaluator may encounter four situations:

• Strategy and planning papers explain why certain objectives have been rejected and identify reasons to support this decision.
• Working papers (interim versions, notes and various correspondences) provide elements to pinpoint rejected objectives and may justify the choices made (in principle, they always include the positions of Member States and the response of the European Commission).
• The documentation available does not explain the choices made, but interviews with the decision-makers provide relevant information.
• The evaluators have not collected any written or verbal information relevant to the explanation of the choices made.

In the first three situations, the evaluators should investigate which of the objectives were planned but eventually rejected, and the reasons for rejection.

In the last situation, the identification of the objectives will have to be made by the evaluation team, or with the help of external expertise. The propositions developed this way can be supported by:

• Information and analyses stated in the available documentation (for example, about the beneficiary country's situation or other donor commitments)
• Examples from similar countries

Whatever the situation, the outcome is the answer to the following question:

Why such an objective has been rejected?

Several questions of this type may be needed at each decision-making point.

Before the construction of the diagram, the evaluation manager or, failing that, the evaluation team should check the quality of the sources of information. If this verification reveals poor or unreliable sources, the construction of the diagram should be abandoned.

Useful information, written or verbal, should not be identified in a too limited way, nor in a comprehensive way. Indeed, the identification of appropriate information for any type of evaluation can only be stated in general.

Written information
Usually, written information can be found in four types of documents.

• Strategy papers and programmes can provide information about the lessons learned from previous implementations, the political, economic and social context, and the interventions of Member States and the main donors.
• The preparatory documentation (meeting reports, notes at the end of preparatory missions, correspondence, internal notes or notes between services) may provide explanations about the priorities chosen.
• Country/thematic/sector-based and project evaluations can be valuable in terms of lessons learned.
• The European Council and Commission's more general documents (conclusions, recommendations, reports), as well as the international treaties and agreements often display the contextual elements, lessons learned and priorities which are known to the designers of strategies and programmes.

Verbal information
To complement the written information, the evaluator can ask the authors and contributors of the main documents (strategy papers and programmes) to explain the reasons for their choices.

Written information
The following list, relating to the kind of available documents and the nature of their information, is not intended to be comprehensive.

• The final version of the Country Strategy Paper (CSP) indicates the priorities and the final programming; sometimes, it describes the lessons learned from previous implementations, the economic and social problems of the country, and the priorities of the beneficiary government; it usually includes reference to co-operation agreements of the Member States and main donors. The successive versions of the CSP may reveal the priorities and programming intentions which were rejected.
• Meeting reports and notes following preparatory missions may provide justification for the determination of the priorities and programming.
• Documents produced during the drafting of the documentation, correspondences, internal notes, and notes between services may specify and explain the resolution of the priorities and programming.
• Documents describing the reactions of Member States to the drafts addressing the choices made, and the need for explanation, can highlight the types of questions which were debated.
• The European Commission's compulsory reply to the Member States' questions shows the justification for the choices.
• Documents collecting the partner government's reactions to the projects may explain governmental priorities.
• Thematic, sector-based and project evaluations may include lessons learned and recommendations, which explain the choices made.
• Mid-term revisions are supposed to review the analyses underpinning the selection of the priorities and programming.
• Specific bilateral and multilateral agreements (co-operation, association) and reports from joint monitoring yield details about the common or specific priorities of the two partners.

Most of these documents can be found for the strategies established after 2001. Prior to this date, the situation is more challenging because the documents are not routinely made available, even when the relationship between the evaluators and the Commission's services is productive.

Verbal information
In addition to - or in substitution for - written information, the authors and contributors to the main documents (strategy and programming papers), as well as the decision-makers responsible for the drafts in process, may be valued informants, providing that they are available and in post for a significant period in the head office (DG Dev and DG Relex, EuropeAid), or in one of the delegations.

Collection of written information
Written information should be collected during the preliminary stage of the country evaluation from the relevant services of the Commission, such as the DG Relex and DG Dev (country and regions), and in the European Commission's Delegation in the country under consideration.

Collection of verbal information
The main interlocutors are the managers of the services previously described. If they are unavailable, former managers can also be interviewed if they can be easily contacted.

If the evaluation managers have not identified a list of information sources, the evaluation team should formulate its conclusions about the quality of the sources in terms of quantity, relevance, reliability and accessibility. This judgement is presented to the managers, who take the final decision.

Preliminary selection of the relevant information is carried out on the basis of the objectives diagram. It requires:

• The establishment of one or more timelines, describing the successive events and/or information included in the drafting process of the strategy and the programme
• The selection of texts from the documentation collected, relative to the choice of the objectives and the scheduled assistance process, or relative to the factors influencing such choices
• The classification and the construction of an index for the texts

Following the analysis of the texts, additional information resulting from the documentation collected may be required, or new documents could be requested and investigated (this method yields uncertain findings).

Provisional answers to the evaluation questions are formulated on the basis of the information collected. Some of the questions may not be answered at this stage.

A table of questions can illustrate the results of the information analysis and take the following shape:

Table A: results of the collected information analysis

Code for the decision-making	Question (wording of the choice)	Answer (justification of the choice)	Identification of the original text

In Table A, the columns gather data concerning:

• The decision-making point: determination of a code highlighting the decisions chain and the successive decision-making point. This code can be the date when the choice has been made.
• Questions about the justification of the decision: each decision-making point corresponds to several questions about the rejected and selected objectives
• Answers to questions: justification of the choices determined by the strategies and programmes
• The identification of the original texts refers to the texts which have been selected and coded

The objectives diagram supports the decisions chain. It identifies four (sometimes five) decision-making points dealing with:

• The overall objective
• The first row intermediary objectives
• The secondary row intermediary objectives
• Operational objectives

Most of the time the drafting of strategies and programmes is not strictly and exclusively driven by such a rationale and is, in fact, often empirical. Implicitly or explicitly, the designers of the strategy and programme begin with an overall objective. Thereafter, they examine which means at the disposal of the European Commission are able to achieve this objective. The successive choices can be synthesised in two points:

• The choice of the overall objective
• The choice of the fields and the intervention processes

Preceding each point, and on both sides of the decisions chain, flows of inputs identified from the analysis of the written and verbal information collected, can be found.

At the left of the sketch, a magnifier explains the decision's outcome:

• Selected objectives
• Rejected objectives

A global fishbone shaped diagram is thus completed through detailed diagrams corresponding to each decision-making point.

Sketch of the moment of the decision-making: example of first row intermediary objectives

The decisions and their explanation must be confirmed by the main actors responsible for the drafting of the strategy and the programming, including the European Commission's services (head office and delegations), the representatives of the other stakeholders (Member States, NGOs, etc.), the usual interlocutors in beneficiary countries and/or their government.

The observations of the respondents can be recorded in the following table.

Table B: Results of the testing of the temporary diagram

Date and code for the decision-making	Decisions made according to the temporary diagram	Justification based on the temporary diagram	Observations by the decision-makers consulted

In Table B, the columns gather data concerning:

• The decision-making point: identification of the decision-making within the decisions chain (identical code to the first column of Table A)
• Decisions made according to the temporary diagram: determination of the selected and rejected objectives which have been subjected to the questions presented in the second column of Table A
• Justification based on the temporary diagram: answers to the questions justifying the decisions set out in the strategies and programmes third column of Table A
• Observations by the decision-makers consulted: approval (with or without reservations) or rejection of the decisions and their justification

If the observations of the decision-makers consulted are not critical, the temporary diagram is corrected incorporating the observations. This operation produces the final diagram.

The process of testing of the temporary diagram may question some of its parts when the justifications do not illustrate the real strategic and programming drafting process.

In this case, the information should be reviewed and augmented by another consultation round. The evaluators and the evaluation managers should decide whether they need to construct a new temporary diagram and test it. Indeed, the observations collected during the test may be sufficient to avoid another consultation and to establish the final diagram directly.

The final and temporary diagrams have the same shape (a main diagram and the sketch of the point of decision-making). The final diagram includes an explanatory table about the analysis of the information collected.