What are the most frequent limitations?
A highly innovative activity
If the question concerns an innovative instrument, activity or procedure, the following difficulties may arise:
- It is difficult to define the terms of the question without ambiguity.
- There is a lack of expertise to understand the cause-and-effect mechanisms.
A very recent activity
If the question concerns a recently implemented activity:
- The activity has not yet produced observable effects
- The informants have not yet stabilised their opinions.
If the question concerns activities in which there are or were managerial weaknesses, the following difficulties may be encountered:
- The monitoring data and reports are inadequate or unreliable.
- The managerial difficulties have generated conflicts that limit access to certain informants or cause those informants to bias their answers.
In case of a strong suspicion of illicit or illegal practices, it is preferable to postpone the evaluation question for later and to start with an audit.
A scope that is too complex
If the question concerns a multi-dimensional or multi-sectoral scope, the following difficulties may be encountered:
- In view of the available time and budget, there are too many data to collect, informants to meet and analyses to perform, and they are too dispersed.
A far-reaching impact
If the question concerns a far-reaching impact which is connected to the evaluated activity through a long chain of causes and effects, then the following difficulties may be encountered:
- There are so many external factors and they are so influential that it becomes impossible to analyse the contribution of the intervention.
An intervention that is too marginal
If the question concerns a very small activity compared to other neighbouring policies or to the context, the following difficulties may be encountered:
- " The evaluated activity does not attain the critical mass that would allow an analysis of its contribution.