ISA² Interim Evaluation
As the ISA² Programme reached its midway point, the European Commission has launched an interim evaluation of the programme. The interim evaluation is a legal obligation coming from the ISA² Decision establishing the programme. It aims to assess how well the ISA² Programme has performed since its start in 2016 and whether its existence continues to be justified. The Commission will use the evaluation results to improve the implementation of ISA² and its eventual successor programme(s) to bring more value to its stakeholders.
To learn more about the process of the interim evaluation, read the Evaluation Roadmap.
Interim Evaluation Objectives
The European Commission contracted the Centre for European Policy Studies (CEPS) to conduct an independent interim evaluation study which has two main objectives:
- Evaluating the implementation of the programme by considering seven evaluation criteria:
- Relevance - the alignment between the objectives of the programme and the current needs and problems experienced by stakeholders;
- Effectiveness - the extent to which the programme has achieved its objectives;
- Efficiency - the extent to which the programme’s objectives are achieved at a minimum cost.
- Coherence - the alignment between the programme and comparable EU initiatives as well as the overall EU policy framework;
- EU added value - the additional impacts generated by the programme, as opposed to leaving the subject matter in the hands of Member States;
- Utility - the extent to which the programme meet stakeholders’ needs;
- Sustainability - the likelihood that the programme’s results will last beyond its completion.
- Identifying areas of potential improvement and devising recommendations to improve the functioning of the programme and bring more value to its partners.
As part of the interim evaluation study, a number of consultation activities was carried out, in line with the Commission’s Better Regulation agenda.
All the planned consultation activities have been completed. Feedback from all relevant stakeholder groups was collected using different channels, leading to 129 individual responses. This included:
- A total of 20 targeted interviews were conducted with relevant stakeholders as listed in the table below between December 2018 and January 2019.
- Targeted online surveys were made available via EUSurvey, designed to match the content of the targeted interviews in order to maximise the number of stakeholders reached. The surveys were open for feedback between 14 December 2018 and 23 January 2019. During this period, 80 respondents provided their input.
- A public consultation was available online in 23 EU languages (from 7 December 2018 to 1 March 2019) for anybody interested to share views and opinions on the performance of ISA². Overall, feedback was received from 14 stakeholders.
- The ISA² Mid-Term Conference and ISA² Interim Evaluation Kick-Off Workshop offered the opportunity to gather initial input from stakeholders through a written questionnaire that was distributed during both events. At this occasion, 15 answers were received.
Although the number of replies remained rather low, the expertise of involved stakeholders is impressive: 71% of respondents are familiar with digital public services and interoperability to a high extent or to the fullest extent. The low response rate to the public consultation is also telling: it reflects the technical nature of the programme. While interoperability is a key enabler of digital transformation, public administrations tend to address the related work in their back offices.
The European Commission will draw its conclusions based on the CEPS study and present them in the form of a report to the European Parliament and the Council of the EU by 30 September 2019.