The use of indicators is often hindered by the fear of provoking adverse effects. There are several types of adverse affect:
- The skimming-off or creaming effect,
- Convergence to the average,
- Unanticipated effects where results are subordinated to indicator scores.
Skimming-off effects can occur when the performance of training and employment services organisations is measured by the placement rate of beneficiaries. To obtain a better placement rate for their beneficiaries, it is in the organisations' interests to recruit people in the best possible situation who also meet the eligibility criteria. The operators therefore tend to "skim off" or favour those beneficiaries whose employability is higher. This effect is undesirable because it focuses assistance on those who are in relatively less need.
An example of how indicators caused a reduction in differences by a convergence towards the average is given in Box Adverse affects of a performance management indicator system: convergence towards the average rather than excellence . An indicator can also encourage behaviour leading to sub-standard performance. This occurs when the indicator rewards undesired results or when the system causes the operators to work for the indicator rather than for the result.
Adverse effects inevitably appear after two or three years of functioning of a system of indicators, no matter how well it is designed. These undesirable effects are generally not foreseeable and their possible appearance should not be an argument for refusing to measure performance. It is possible to minimise adverse effects, either by amending the indicator causing the problem, or by creating a procedure for interpretation of the indicator by expert panels. It is therefore important to watch out for adverse effects and to correct the system when they appear.
Creating performance incentives
There are several ways of using indicators to promote an improvement in operators' performance. These include:
- Operators with poor performance receive specific technical assistance to help them progress. If the situation does not improve, the budget is restricted. This method works on the principle that it is not the mistake that must be penalised but rather the inability to correct mistakes.
- Operators with the best performance are granted greater autonomy and are controlled less.
- Operators with the best performance receive support for presenting their outputs and results to the general public.
- Operators who did not perform well enough are disqualified from the selection procedures for future projects.
- Operators with the best performance are offered additional funds (see Box The EU Performance Reserve, 2004).