ISA² Interim Evaluation
/isa2/file/isa2interimevaluation-visual-banner-2400x450-01jpg_enisa2_interim_evaluation-visual-banner-2400x450-01.jpg
As the ISA² Programme reached its midway point, the European Commission launched an interim evaluation of the programme. The interim evaluation is a legal obligation coming from the ISA² Decision establishing the programme. It aimed to assess how well the ISA² Programme performed since its start in 2016 and whether its existence continues to be justified. The Commission will use the evaluation results to improve the implementation of ISA² and its eventual successor programme(s) to bring more value to its stakeholders.
To learn more about the process of the interim evaluation, read the Evaluation Roadmap.
Interim Evaluation Objectives
The European Commission contracted the Centre for European Policy Studies (CEPS) to conduct an independent interim evaluation study which had two main objectives:
- Evaluating the implementation of the programme by considering seven evaluation criteria:
- Relevance - the alignment between the objectives of the programme and the current needs and problems experienced by stakeholders;
- Effectiveness - the extent to which the programme has achieved its objectives;
- Efficiency - the extent to which the programme’s objectives are achieved at a minimum cost.
- Coherence - the alignment between the programme and comparable EU initiatives as well as the overall EU policy framework;
- EU added value - the additional impacts generated by the programme, as opposed to leaving the subject matter in the hands of Member States;
- Utility - the extent to which the programme meet stakeholders’ needs;
- Sustainability - the likelihood that the programme’s results will last beyond its completion.
- Identifying areas of potential improvement and devising recommendations to improve the functioning of the programme and bring more value to its partners.
Read the evaluation study by CEPS (EN) or its executive summary (EN/FR).
Consultation activities
As part of the interim evaluation study, a number of consultation activities was carried out, in line with the Commission’s Better Regulation agenda.
Feedback from all relevant stakeholder groups was collected using different channels, leading to 129 individual responses. This included:
- A total of 20 targeted interviews were conducted with relevant stakeholders as listed in the table below between December 2018 and January 2019.
- Targeted online surveys were made available via EUSurvey, designed to match the content of the targeted interviews in order to maximise the number of stakeholders reached. The surveys were open for feedback between 14 December 2018 and 23 January 2019. During this period, 80 respondents provided their input.
- A public consultation was available online in 23 EU languages (from 7 December 2018 to 1 March 2019) for anybody interested to share views and opinions on the performance of ISA². Overall, feedback was received from 14 stakeholders.
- The ISA² Mid-Term Conference and ISA² Interim Evaluation Kick-Off Workshop offered the opportunity to gather initial input from stakeholders through a written questionnaire that was distributed during both events. At this occasion, 15 answers were received.
Although the number of replies remained rather low, the expertise of involved stakeholders was impressive: 71% of respondents were familiar with digital public services and interoperability to a high extent or to the fullest extent. The low response rate to the public consultation is also telling: it reflects the technical nature of the programme. While interoperability is a key enabler of digital transformation, public administrations tend to address the related work in their back offices.
Read more in the Synopsis report of the consultation activities.
/isa2/file/plannedconsultationactivitiesjpg_enplanned_consultation_activities.jpg
/isa2/file/evaluationtimelinepng_enevaluation_timeline.png
Final results of interim evaluation
The European Commission has drawn its conclusions based on the CEPS study and adopted the interim evaluation results in the form of a report to the European Parliament and the Council of the EU on 23 September 2019. Overall, the interim evaluation concludes the ISA² Programme has been performing well so far, while it also identifies three areas for possible improvement.
Read more in the Report from the Commission to the European Parliament and the Council: Results of the interim evaluation of the ISA² Programme.
Next steps
The Commission will take into account the findings and recommendations of the interim evaluation during the further implementation of the ISA² Programme. The results, and in particular the areas for improvement, will also be very valuable for managing the transition to the new Digital Europe Programme.