This Commission staff working document accompanies the report on the results of the interim evaluation of the ISA2 programme.
As a reminder, ISA2 is a European funding programme offering interoperability solutions and common frameworks for European public administrations, businesses and citizens. It is the fifth in a series of European instruments fostering interoperability in the European public sector since 1995. It follows the ISA programme, which terminated in 2015, and covers the 2016-2020 period.
ISA2 was established by Decision (EU) 2015/2240 of the European Parliament and of the Council (‘ISA2 Decision’), which legally requires the European Commission to carry out an interim evaluation of the programme by 30 September 2019, and to communicate the results.
Overall findings
The Commission noted that although the evaluation was able to draw robust conclusions for all evaluation criteria based on the collected data, there were two limitations:
(1) Timing: as this is an interim evaluation, the analysis covered only the first three years of the programme (2016, 2017 and 2018). The ongoing ISA2 actions are expected to produce further results in the coming years, which will influence the programme’s overall achievement of objectives.
(2) Reach: ISA2 is a very technical programme, and mainly involves interoperability experts. This explains the rather low participation in the stakeholder consultation activities. Another reason stems from the stringent confidentiality and data protection rules: without their specific consent, the evaluation team could approach ISA2 solution users only through the action owners acting as intermediaries.
Based on data collected from 129 consulted stakeholders, extensive desk research, and expert assessments, the interim evaluation confirms that ISA2 performs well in all the evaluation criteria.
Relevance: the original needs and problems that the programme intended to address are still fully relevant and the objectives of ISA2 address them. One additional need that deserves special attention is the request to ensure more collaboration and exchanges with regional and local administrations in order to increase awareness of interoperability and the take-up of ISA2 solutions at the sub-national level.
Effectiveness: the results that ISA2 has achieved so far are aligned with its objectives. There is one specific ISA2 objective where the evaluation found that the programme on its own is less effective: the development of a more effective, simplified and user-friendly e-administration at the national, regional and local level.
Efficiency: the programme’s implementation is progressing as planned, all actions being either on track or close to achieving the planned level of work. Nevertheless, the heterogeneity of performance indicators makes it difficult to draw conclusions about its overall cost-effectiveness.
Coherence: substantial synergies and limited overlaps among ISA2 actions paint a positive picture of the programme’s internal coherence. This strong internal coherence is in line with the second recommendation of the final ISA evaluation, which called for taking a holistic approach to interoperability within ISA2.
Synergies were established with the Connecting Europe Facility, Horizon 2020 and the Structural Reform Support Programme.
Improvements
The interim evaluation identified several potential measures that may improve the performance of ISA2 and its successor instruments. It is up to the programme’s management to assess the feasibility of these actions and put forward specific recommendations, which the Commission could then put into practice in the future.