Specify the scope of the question
The scope of the question can be the entire intervention or a particular dimension of its design or implementation, for instance:
- Has the intervention helped to generate effect X as expected? (entire intervention)
- Has the actors' participation in the design of the intervention helped to generate effect X more successfully (design modality)
- Have the measures taken to ensure coordination with the other donors helped to generate effect X more successfully? (implementation modality)
- Has the choice of sector budget support helped to generate effect X more successfully (funding modality).
A question concerning both the entire intervention and all its effects would be very difficult to answer, except in the case of a simple intervention,. At least one of the two elements has to be specified.
How to define the scope of the question
Questions on design and implementation (examples)
Link the question to the intervention logic
The intervention logic identifies the main expected effects (outputs, results and impacts). Which effect does the question concern? For example:
- Has the intervention helped to generate effect X as expected?
- Has the intervention helped to generate effect X for the poorest groups?
- Has the intervention increased the poorest groups' chances of obtaining effect X?
If the question concerns a short-term result or specific impact, it will be more easily evaluable than if it concerned a global impact. Conversely, a question on a global impact will be more useful for decision-makers at a strategic level.
It is possible to ask a question that is not connected to the intervention logic, for example:
- Has the intervention generated unexpected effects for the people affected?
Questions concerning the whole range of expected effects are likely to be too vast for the evaluation team to be able to give a satisfactory answer.
How can a question be specified in relation to the intervention logic?
How to reconstruct the intervention logic?
Link the question to an evaluation criterion
The question is drafted with an evaluation criterion in mind (e.g. relevance, effectiveness, efficiency, sustainability, impact, Community value added, and coherence/complementarity), unless it is designed only to further knowledge and understanding.
Questions on efficiency and sustainability are asked more rarely, partly because they are difficult to answer. Yet they are often the most useful.
Specify the family of evaluation criteria
Questions belonging to the different families of evaluation criteria (examples)
Families of evaluation criteria