Communication is an important part of the evaluation process. It is better to treat the communication task as continuous: an opportunity for dialogue and the accumulation of understanding rather than put all communication efforts into one big dissemination exercise after a final report is delivered. Communication should therefore include:
- Improving awareness of the evaluation underway;
- Providing feedback on interim findings;
- Circulating and managing feedback on draft reports and outputs (e.g. data collection instruments); and
- Communicating evaluation findings and conclusions.
Improving awareness of the evaluation underway
Once the evaluation team has been engaged it is useful to provide information to stakeholders on the timetable and process. The inception period should be used as an opportunity to both explain the planned approach and to canvas opinions on the usefulness of the evaluation questions and the likely success of what is being proposed. In addition to formal information provided to stakeholders perhaps through the steering committee, general information to the public and beneficiaries perhaps in the form of press releases or information on websites can also be a useful way to prepare the ground for the evaluation.
Providing feedback on interim findings
The communications of interim findings pose major challenges. On the one hand stakeholders are likely to have a keen interest in early findings particularly if they suggest that the ultimate findings will be critical. At the same time the evaluation team may be hesitant about inferring major conclusions and nervous about the strength of the evidence base for their observations. They may (but should not) also view the production of interim findings as little more than a bureaucratic necessity (it is not unusual for interim reports to trigger interim payments). It is best if attention is given in the inception report to the likely scope and content of interim findings and the method and extent to which they will be circulated. Interim findings can provide useful feedback on process and implementation (e.g. suggest changes in procedure) and help increase the engagement of stakeholders and those involved both in the programmes and in the evaluation.
Circulating and managing feedback on draft reports and findings
Producing the draft final report is often a difficult stage both for evaluators and stakeholders. What has previously been an abstract anticipation of outputs now becomes real and sometimes threatening or disappointing. Stakeholders, especially those with programme management responsibilities may be tempted to discredit findings they do not like. Evaluators for their part may construct arguments on limited evidence or be insensitive to the political import of what they present. Producing a final report that is acceptable to the evaluation team and the commissioning authority and respected by stakeholders who have been engaged in the process is a major challenge and requires a good deal of time. The following suggestions may facilitate the process:
- The structure of the report should be agreed as early as possible.
- The Steering Committee should be the main forum for discussion of the draft.
- The contracting authority should avoid the temptation to overly influence the formulation of conclusions and recommendations. Rather they should challenge the evaluation team to justify their conclusions and recommendation on the basis of the evidence presented.
- Sufficient time should be given for written comments.
- The contracting authority should take responsibility for the circulation of the report and compiling feedback.
Communicating evaluation findings
Evaluation is of no consequence unless the findings are communicated. The principal form of communication is a written report. Whilst the appropriateness of the particular means of communication will vary there are a number of good practices:
- The written report should be clearly written and concise. One hundred pages including an executive summary are normally sufficient. Detailed evaluative evidence such as case studies and quantitative analysis should be presented in annexes or made available separately.
- The report should include an executive summary of 5-10 pages written in a style suitable for policy makers.
- The links between the conclusions and the analysis of evidence should be clear.
- The drafting of the report should indicate the basis for the observations made: the evaluation evidence or a combination of evidence and the evaluator's opinion.
- The report should include a description and assessment of the method used that is sufficiently detailed and self critical to enable the reader to judge the weight of evidence informing the conclusions.
- Use should be made of tables and diagrams where they improve the presentation of findings.
- Reference should be made to good practice examples of interventions to illustrate the arguments being made but evaluation reports should not take the place of good practice guidance. Pressure on evaluators to produce good news stories is often counterproductive: such results are viewed with suspicion by public and policy makers alike.
- The recommendations made should be clear in the follow-up action that is required.
Channels for communicating evaluation findings and reaching users
Those responsible for commissioning and undertaking the evaluation should ensure that the results are communicated and used. Potential users, from policy makers through beneficiaries to the general public, need to be identified and the most appropriate channels of communication selected.
Evaluation reports are normally published, increasingly on the internet. Written reports should also include more popular outputs for news media to take up. Many programmes produce their own newsletters and these provide another opportunity for dissemination. Verbal presentations to the Steering Committee and other stakeholders (e.g., in specialised workshops) are also useful.