Interpreting and comparing indicators

Situations exist in which indicators speak for themselves, but these are exceptions. In general, indicators have to be interpreted by means of the relevant comparison or breakdown. In one example, the comparison of three indicators showed that the training financed with EU funding did not reach the long-term unemployed as it should have (see Box Comparing several indicators to reveal a phenomenon )

Whenever possible, it is useful to compare programme indicators (for example, km of roads (motorways) built) with the appropriate context indicators (for example, baseline km of motorway in the region). This type of information can be presented in a concise way by means of derived indicators that express the programme outputs or effects as a percentage of the context indicator.

In order to be useful in evaluation work indicators need to be used in conjunction with qualitative findings. To interpret indicators, it is necessary to consider the context as a whole, the factors which help to facilitate or hinder the performance of the programme, the rationales of the programme, and the process of implementation.

One technique consists of asking an expert panel, to examine the combined quantitative and qualitative elements of the situation, to interpret the performance measures (see Box Expert panels to assess University Research Centre performance ).

The interpretation of indicators often makes use of comparisons between the performance of two interventions. This type of comparison mainly concerns efficiency indicators, for example: cost of a trainee's training; cost of a job created; cost of a home connected to the sewerage system. However, comparisons are not always acceptable for programme operators and managers. In particular, performance comparisons will seem unfair to managers who work in the more difficult contexts. To make comparisons acceptable, the comparison procedures must be considerably refined with a view to taking account of the additional challenges of interventions taking place in more difficult environments.

An example of how this issue was addressed in the education domain given in Box Example of comparison of performance accepted by the operators . The technique consists of reconstituting, for each intervention to be compared, a group of several similar interventions implemented in other regions in an environment with a similar degree of difficulty. Comparisons were made with the average of the interventions of this "benchmark group".

Last update: 20/02/2008  |Top