Evaluations of the 2007-2013 programming period

Additional tools

 

• Summary: The main purpose of this contract was to 1) gather and quality assess physical data reported by Managing Authorities from 2007-2013 in their Annual Implementation Reports (in particular 21 core indicators and specific indicators), 2) assess the reporting on major projects in the annual reports and 3) assess the feasibility of gathering financial data broken down by priority theme to the NUTS2 and 3 levels of region.
• Deliverables:
  • Final report
  • Executive Summary
  • WP0 Database 1 – Selected Core Indicators (5 000 values) used by DG REGIO to calculate aggregate achievements for core indicators
  • WP0 Database 2 - full database including all core indicators and programme specific indicators (10 000 values) - Database guide
  • WP0 Database 3 - Major Projects
  • WP0 Table on availability of Expenditure data at NUTS3 level by priority theme and Explanatory Note
  • Terms of Reference

• Summary: Work Package 1 includes the Synthesis of the ex post evaluation of ERDF and Cohesion Fund support for the programming period 2007-2013, covering all related work packages. In addition, the work package includes several intermediate reports such as, for example, macroeconomic developments in the EU during 2007-2013, regional development trends in the EU during 2007-2013, and summary of the seminar on job creation.
• Deliverables:
  • Synthesis report ex-post evaluation 2007-2013
  • Macroeconomic situation 2007-2013
  • Regional development trends in the EU
  • Country reports and data
    • Austria: report - data
    • Belgium: report - data
    • Bulgaria: report - data
    • Croatia: report - data
    • Cyprus: report - data
    • Czech Republic: report - data
    • Denmark: report - data
    • Estonia: report - data
    • Finland: report - data
    • France: report - data
    • Germany report - data
    • Greece report - data
    • Hungary report - data
    • Ireland report - data
    • Italy report - data
    • Latvia report - data
    • Lithuania report - data
    • Luxembourg report - data
    • Malta report - data
    • The Netherlands report - data
    • Poland report - data
    • Portugal report - data
    • Romania report - data
    • Slovak Republic report - data
    • Slovenia report - data
    • Spain report - data
    • Sweden report - data
    • United Kingdom report - data
    • Interreg
  • Summary of Expert seminar on the use of Cohesion policy funding to support growth and job creation in the EU (30 January 2015)
  • Inception report
  • Terms of Reference

• Summary: This evaluation comprises the analysis of 50 OPs representing 65% of the support to SMEs and outlines the mechanisms and context where the OPs were implemented.
  
  The study offer an in-depth examination of eight case studies of selected OPs and a theory-based impact evaluations of three policy instruments.
  
  Key findings:
  • The ERDF provided a significant source of funds which helped SMEs to cope with the crisis in particular in those regions most severely affected.
  • The ERDF enabled SMEs to survive or preserve pre-crisis levels of investment and employment.
  • The analysis shows that more ambitious and structural effects also developed. In these cases, ERDF interventions fostered dynamics of change within targeted SMEs.
  • For some SMEs the contribution of the ERDF accelerated or anticipated investment plans sometimes increasing their magnitude.
  • Selected evidence illustrates that the ERDF played a catalytic role in supporting the strategic investment plans of SMEs, helping to increase profitability and exports.
  • The ERDF triggered changes in the way SMEs do business. Some were visible and measurable (e.g. technologic upgrade), others related to the entrepreneur’s mind set.
  • Behavioural changes were more evident in SMEs which already had longer experience with R&D projects where they stimulated deeper structural effects.
  • When target SMEs were embedded into clusters or local production systems, there was evidence that positive effects spread to other enterprises.
• Deliverables:
  • Terms of Reference
  • 1st intermediate Report: Volume 1 - Volume 2
  • 2nd intermediate Report
  • 3rd intermediate Report: Volume 1 - Volume 2
  • Final Report
  • Executive Summary
  • Case Studies:
    • Czech Republic
    • Denmark
    • France
    • Germany
    • Italy
    • Lithuania
    • Poland
    • Spain

• Summary: Financial instruments are a growing set of tools in Regional and Urban Policy - by 31 December 2012 a total of 816 specific funds had been set up across the EU, with EUR 10.5 billion in capital. This evaluation assesses the rationale, implementation and early evidence of effectiveness of equity/venture capital, loans and guarantees. A key policy question is how the costs (including set up) and benefits (increased investment, production, productivity and jobs) compare between the different financial instruments and with alternative enterprise support instruments such as grants.
• Deliverables:
  • Terms of Reference
  • Final Report
  • Executive Summary

• Summary: Large enterprises play an important role in regional development, but much of the evidence so far questions the impact of public support on large enterprise investment decisions – and their long term commitment to the region of location. This evaluation assesses the rationale, implementation and evidence of effectiveness of Cohesion Policy support in the 8 countries which account for 75% of ERDF support to large enterprises. The evaluation will compare direct effects and wider benefits as well as export-led, FDI-based strategies and development of the indigenous large enterprise base.

• Summary: This evaluation presents main achievements of cohesion policy in the area of transport infrastructure. It also offers assessment of the EU contribution to the comprehensive transport networks and its strength and weaknesses, focusing in particular on the financial analysis underlying the infrastructural projects.
  • At over € 82 billion, transport accounts for the biggest share of cohesion policy allocation (over 30%).
  • This contribution enabled Member States to upgrade or expand their transport networks. By the end of 2013, the managing authorities reported 3,875 km of new roads built and over 23,000 km of roads reconstructed, as well as 269 km of new railroads and 3,136 km of reconstructed railroads.
  • New or enhanced infrastructure provided better connectivity both within and across Member States borders. 47% of roads and 49% of railroads receiving the cohesion policy support were part of the TEN-T.
  • Cohesion policy also stimulated improvement in the way transport interventions are planned (development of transport strategies) and prepared (improvement in quality of major projects applications).
• Deliverables:
  • Terms of Reference
  • Final report bringing together the findings from all evaluation tasks and including catalogue of challenges (problems within the ex-ante financial analysis of major projects and solutions to overcome them)
  • Executive summary
  • Map of road projects supported by Cohesion policy between 2000 and 2013 (and operational by the end of 2014)
  • Map of rail projects supported by Cohesion policy between 2000 and 2013 (and operational by the end of 2014)
  • 10 major project case studies to better understand the implementation context and verify the underlying assumptions (BG, CZ, DE, EE, ES, HU, PL and RO / road, railway and urban transport)
  • 6 Member State case studies to explore the contribution of cohesion policy to the national and EU transport policy (ES, FR, HU, IT, PL and RO)
  • Inception report

• Summary: The evaluation analysed the progress and achievements of Cohesion Policy for environment related infrastructure focusing on two areas: waste management and water. The study assessed to what extent Cohesion Policy funding contributed to Member States' implementation of EU environmental directives and 2) environmental projects co-funded by Cohesion Policy are financially sustainable. In order to answer these questions, a broad review of the portfolio of projects and operational programmes and a more detailed analysis of the quality of the financial analysis for 20 selected major projects were carried out. 10 case studies examined implementation issues and provided a contextual understanding of the financial analysis: 3 projects for waste management (Poland/ Gdańsk, Portugal/ERSUC, Romania/Cluj County) and 7 projects for water (Czech Republic/Brno, Estonia/Kohtla-Järve Area, Greece/Koropiou & Paianias, Lithuania/Vilnius, Malta, Poland/ Żory, Portugal/SIMARSUL).
  
  1. The study found that the support for environment for the programming period 2007-2013 represented about € 46.5 billion and the decided amounts accounted for over 17% of all Cohesion Policy resources. In many EU13 and southern EU15 Member States, Cohesion Policy is one of the main sources of public financing and therefore provided a major contribution to achieving EU water and waste targets. Many of these countries have made broad progress: in the area of waste, the recycling rate increased in almost all EU13 Member States; in the area of water, projects financed by the ERDF and Cohesion Fund improved drinking water supply for at least 4 million EU citizens and contributed to better wastewater treatment for over 7 million EU citizens.
  2. The evaluation found that the financial analysis undertaken as part of the preparation of the examined major projects on water and waste management was of reasonable good quality and provided a sound basis for financial sustainability of those environmental projects.
• Deliverables:
  • Terms of Reference
  • 2 pilot case studies
  • Final Report
  • Executive Summary
  • Case Studies

• Summary: In the 2007-2013 programming period, energy efficiency allocations amounted to EUR 6 billion, a substantial increase from the 2000-2006 period. The evaluation will assess the rationale for types of interventions and early evidence of effectiveness of investments in energy efficiency in public and residential buildings.
• Deliverables:
  • Final report - Annex
  • Terms of Reference

• Summary: The evaluation will establish the nature of co-financed investments in the sector of tourism and culture and explore the rationale that guides their potential and actual contribution to deliver growth and jobs. At least EUR 14 Billion is planned for investment in tourism and culture activities in the period 2007-2013. The study will synthesise outputs and results achieved, and will assess the effectiveness of strategies and interventions in these policy areas.
• Deliverables:
  • Final report - Annex
  • Executive Summary
  • Case Study Interreg Italy-Austria
    • Mini Case Study Interreg Italy-Austria: Transmuseum
    • Mini Case Study Cross-border Italy-Austria Interreg IV Programme Study Ciclovia Alpe Adria Radweg
  • Regional Case Study Madrid
    • Mini Case Study Madrid: Centro de Arte Alcobendas
    • Mini Case Study Madrid: Matadero
  • Case Study Malta
    • Mini Case Study Malta: Grant Scheme for Sustainable Tourism Projects by Enterprises
    • Mini Case Study Malta: The Restoration and Rehabilitation of Historical Fortifications of Malta and Gozo
  • Podlaskie Regional Case Study
    • Mini Case Study Podlaskie: Centre for Education and Muslim Culture of Polish Tatars in Kruszyniany
    • Mini Case Study The Opera and Philharmonics of Podlaskie
  • Regional Case Study Puglia
    • Mini Case Study Buy Puglia
    • Mini Case Study Puglia Sounds
  • Regional Case Study Rhone-Alpes (France)
    • Mini Case Study Rhone-Alpes: Création d’un espace de promotion/communication touristique sur l’Ain sur le site du Parc des Oiseaux
    • Mini Case Study Rhone-Alpes: Maison de la Lone in Saint Pierre de Boef leisure park
  • Interim Report - Annexes
  • Inception report
  • Terms of Reference

• Summary: The evaluation will establish the nature and objectives of co-financed investments in the fields of urban development and social infrastructures and assess the extent to which they were delivered through integrated strategies or not. The overall aim of the evaluation is to explore the rationale that guides investments in these fields and their potential and actual contribution to deliver growth and jobs.
• Deliverables:
  • Terms of Reference
  • Final Report - Annexes
  • Executive Summary

• Summary: The evaluation will establish what results have been achieved via co-operation programmes across Europe, with particular focus on co-operation in the field of research, technology and innovation, environmental protection and enhancement, and capacity building. It will assess the results of knowledge transfer across regions from the INTERREG IV C Programme and analyse to what extent ETC programmes contributed to the Jobs and Growth agenda.
• Deliverables:
  • Final Report
    • Pilot case study: Interreg IVA Northern Ireland, Border Region of Ireland and Western Scotland
    • Case study: Hungary-Slovakia Cross-border Cooperation Programme 2007-2013
    • Case study: INTERREG IV A Deutschland – Nederland
    • Case study: Interreg IVA France (Channel)-England
    • Case study: Baltic Sea Region programme
    • Case study: Atlantic Area programme
    • Case study: Interreg IVC
    • Case study: Spain-Portugal
    • Case study: South Baltic Cross-Border Cooperation Programme
    • Case study: Case study: Romania – Bulgaria Cross Border Cooperation Programme 2007-2013
    • Case study: Interreg IVA North
    • Case study: Interreg IVA Saxony-Czech Republic
  • Inception report
  • Terms of Reference

• Summary: The main purpose of this evaluation is to analyse the effectiveness and efficiency of cohesion policy implementation during the 2007-2013 period and to identify the main underlying success factors and areas in need of development. The evaluation covers cohesion policy programmes financed by the European Regional Development Fund (ERDF), the European Social Fund (ESF) and the Cohesion Fund (CF).
• Deliverables:
  • Final report - Annex
  • Inception report
  • Terms of Reference

• Summary: The study collected and mapped information on the regional breakdown of the ERDF and CF invested through more than 300 Programmes, (Convergence Objective, RCE Objective and cross border programmes under the ETC Objective). The aim was to identify cumulative allocations to selected projects and expenditure at NUTS3 and NUTS2 level (where available) broken down by the 86 priority themes and to make estimates, based on a elaborated methodology, where the data was not available. 2007 – 2013 database was consolidated with 2000 – 2006 data, creating a unified database for the last two programming periods at NUTS2 level. Further consolidation of 1994-1999 expenditure at NUTS2 level was considered.
• Deliverables:
  • Final report
  • Executive summaries
  • Database of the cumulative allocations to selected projects and expenditure at NUTS3 level broken down by the 86 priority
  • Database of the cumulative allocations to selected projects and expenditure at NUTS2
  • Integrated database of allocations and expenditure for 2000-2006/2007–2013
  • Maps
  • Terms of Reference

• Summary: This report provides an assessment of the programmes implemented under EU cohesion and rural development policies during the programming period 2007-2013. The impacts of these programmes at country level, and on the EU economy, are analysed based on a set of simulations conducted with QUEST III, a model developed by the Directorate General for Economic and Financial Affairs of the European Commission.
  Overall, this analysis indicates that the policies for cohesion and rural development yield high value for money. As expected from policies aimed at supporting long-term sustainable development, the impact on GDP per euro invested increases steadily over time. Furthermore, the results indicate that the Union efforts to allocate resources to these policies generate a common benefit for all the members of the EU, especially in the medium and long run.
• Deliverables:
  • Final report

• Summary: This analysis provides an assessment of the programmes implemented under EU cohesion policy during the programming period 2007-2013. The impacts of these programmes are analysed at regional level, based on a set of simulations conducted with RHOMOLO, a regional model developed by the Joint Research Centre of the European Commission. The results of the study indicate that the impacts of cohesion policy are positive and significant, particularly in regions which are the main beneficiaries of cohesion funds. Over the medium to long term, the impacts on regional development range from 0.2% to 9% of estimated additional GDP across all regions in EU27. When compared with the analysis at country level (such as in WP1a with QUEST), the current study illustrates the added value of an analysis at regional level: depending on the stage of regional development and on the amount and timing of resources allocated, the potential of cohesion investments may differ markedly in terms of impacts on regions within countries.
• Deliverables:
  • Final report

• Summary: These counterfactual impact evaluations estimate the effects of the varying intensities of the EU Funds (sum of structural and cohesion funds) on a number of regional growth variables in the EU-27 regions during the period 1994 to 2013.
  
  The results show a positive impact in terms of annual per-capita GDP growth due to the higher intensity of EUF in the Objective 1 /Convergence regions. This is compared to a counterfactual status of receiving the lower average intensity of the non-Objective 1 / non-convergence regions. The impacts on gross fixed capital formation and employment rate changes are also positive but not statistically significant.
  
• Deliverables:
  • Work package 14 c and 14d - Executive Summary
    • Final technical report - Measuring the impact of Structural and Cohesion Funds using the Regression Discontinuity Design in the EU15 1994-2011
    • Final technical report - Measuring the impact of Structural and Cohesion Funds using regression discontinuity design in EU27 in the period 1994-2011
    • Final technical report - Propensity score matching

The objective of this network of experts is to synthesise evidence on the performance of Cohesion policy in the 2007-2013 period by examining the physical and financial performance of the operational programmes and evaluation and other evidence available. Every year the expert network produces country reports on the achievements of cohesion policy and on a selected theme. All reports look at the physical and financial performance of the operational programmes and analyse evidence coming from evaluations undertaken in the Member States.

Another task of the network is a peer review of selected evaluations. The idea is to encourage the relevant authorities to carry out more and more rigorous evaluations. The review looks both the relevance of the evaluations as well as the techniques applied to produce meaningful and reliable results. A panel of a small number of high-level experts acts as the peers in this process.

2013 Reports

• Note on the provisional results of the peer review process

For 2013, the network of experts produced 27 policy papers (one for each country) on the topic of job creation as an indicator in ERDF programmes. The main findings from the country reports are summarized in a synthesis report.

• Synthesis Report
• Policy papers

The network of experts also produced for every Member State a report on the achievements of cohesion policy. The main findings are summarized in a synthesis report:

• Synthesis Report and executive summaries (DE-EN-FR)
• Country Reports on achievements of cohesion policy

2012 Reports

For 2012, the network of experts produced 27 policy papers (one for each country) on the topic of financial instruments. The main findings from the country reports are summarized in a synthesis report.

Financial engineering

• Synthesis report
• Executive Summary
• Policy papers
• Literature review

The network of experts also produced for every Member State a report on the achievements of cohesion policy. The main findings are summarized in a synthesis report:

• Synthesis Report and executive summaries (EN, FR, DE)
• Country Reports on achievements of cohesion policy

2011 Reports

For 2011 the network of experts produced for every Member State a report on the achievements of cohesion policy and a policy paper on renewable energy and energy efficiency in residential housing. The network also identified examples of good practice in evaluation (interesting methods, good data, etc.). The main findings from all country reports are summarized in the synthesis reports which contain comparative analysis and trends across the EU.

• Synthesis of national reports 2011
• Executive Summary
• Country Reports on achievements of cohesion policy : Country reports - Tables
• Synthesis report on renewable energies and energy efficiency of housing
• Policy papers
• Executive summary
• Incentives for developing renewable energy supplies and improving the energy efficiency of housing - Literature review
• Inception report (2011)
• Terms of Reference (2011)

2010 Reports

In 2010 the network of experts produced for every Member State a report on the achievements of cohesion policy and a policy paper on innovation.

• Country reports on achievements of cohesion policy
• Synthesis Report on Achievements of Cohesion Policy - Executive Summary
• Policy papers on innovation
• Synthesis Report on Innovation - Executive Summary
• Feedback from Maltese Managing Authorities
• Inception report (2010)
• Terms of Reference (2010)

The European Observation Network for Territorial Development and Cohesion (ESPON) supports policy making by providing evidence and analysis in the field of territorial cohesion. The current programme has a budget of €47 million: 75% from the ERDF, the remainder from the 31 countries participating (27 EU Member States plus Iceland, Lichtenstein, Norway and Switzerland).

The European council at Poznan in November 2011 agreed that the role of ESPON should be strengthened in future and asked the Commission to conduct "an evaluation of the results of ESPON providing the basis for its future work".

The core of the evaluation was in depth case studies of 12 ESPON research projects - including peer review by academic and territorial policy experts. The case studies were selected to be broadly representative, examining recent projects from each of ESPON's four priorities, covering a range of subjects (e.g. migration, secondary growth poles, portraits of rural regions and indicators of territorial cohesion).

In addition, the evaluation included in depth interviews with monitoring committee members and other key stakeholders as well as discussion of the findings with a scientific committee of experts.

Key findings

Key recommendations include:

• Each research project should have a dissemination strategy throughout its lifecycle. This includes a clear definition of the target audience and a tighter focus on their needs from the terms of reference onwards.
• The ESPON co-ordination unit should take a greater role as knowledge broker, building relationships in the policy community. This is not just about disseminating results, but actively engaging with the policy community to discover what research needs there are which could be filled by future ESPON projects.
• A different administrative set-up should be considered for payment claims - this would free up time for both project leaders and ESPON co-ordination unit.
• Monitoring committee members should take a more strategic role, delegating much of the project work to members with expertise in a given field and/or theESPON co-ordination unit.

Documents

• Final Report - Quality assessment
• Annexes (including case studies)

This study gives an overview of the methods used for assessing different kinds of publicly supported innovation activities throughout Europe. It analyses the advantages and the limits of available methodologies and covers 15 case studies on evaluations of the most popular innovation activities in Austria, Belgium, Denmark, Estonia, Finland, France, Germany, Hungary, Ireland, Italy, the Netherlands, Poland, Sweden, and UK. On this basis, guidance for managing authorities to support their evaluation activities was produced.

• Guidance on evaluation of innovation
• Final report
• Case studies
• Interim Report
  • Synthesis:
  • Appendix A: Literature review
  • Appendix B: Analysis of an online survey of Managing Authorities
• Inception Report
• Terms of Reference

This study analyzed the extent to which cohesion policy interventions have been appropriate and effective in the special territories for the 2000-2006 and 2007-2013 periods. It was carried out for both periods 2000 - 2006 and 2007 - 2013 and was based on analysis of ERDF / CF impact on fifteen NUTS2 regions, and six NUTS3 areas. The study showed that the two financial instruments created a stable long term financial framework in which the regions can operate and develop projects that would have not been possible to be funded otherwise (such as transport or environment infrastructures) or projects that build on local assets (touristic and creative activities, support to cluster for example). It was proved that the instruments had the required flexibility to meet the needs of such challenging regions provided that policy approaches are more strategic and better coordinated through multi level governance arrangements.

• Final report
• The six Case Studies

• Inception Report
• Terms of Reference
• First Intermediate Report
• Second Intermediate Report - overview
  • Castilla-la-Mancha, Spain
  • Sterea Ellada, Greece
  • Itä-Suomi, Finland
  • Highlands and Islands, UK
  • Övre Norrland, Sweden
  • Steiremark, Austria
  • Rhône Alps, France
  • Ślaskie, Poland
  • Centro, Portugal
  • Vzhodna Slovenija, Slovenia
  • The Balearic Islands, Spain
  • Corsica, France
  • Sicilia, Italy
  • Voreio Aigaio, Greece
  • Hovesdstaden including Bornholm, Denmark

The study assesses the contribution of the local development approach to the effective delivery of Cohesion Policy. It covers interventions co-financed by the ERDF in the 2000-2006 and 2007-13 programming periods. Findings from the review of 38 Operational Programmes and the in-depth analysis of 5 regional cases - in Andalusia Region, Berlin Metropolitan Area, Czech Regional Northwest, Puglia Region and West Wales and the Valleys - serve as a base for operational recommendations on how and when local development could be used to deliver Cohesion Policy and how to monitor and evaluate the effects of local development interventions on economic, social and territorial cohesion.

• Final report
• Case Studies : Berlin - Andalucia - West Wales and the Valleys - Northwest Czech Republic - Puglia
• Inception Report
• Terms of Reference
• First Interim Report

The European Commission has launched pilot projects in the fields of early childhood education and microfinance. Evaluation and data-gathering are being carried out in collaboration with the UNDP and World Bank. Here you can follow the progress of the evaluations and the household survey - comprehensive data on the social and economic situations of disadvantaged Roma.

• Inception Report
• Terms of Reference

The study evaluates the potential for current regional policy instruments to contribute to the achievement of growth, jobs and sustainable development. The analysis is based on examination of the Cohesion policy instruments in all 27 Member States including all 246 operational programmes supported by the ERDF and Cohesion Fund, as well as the National Strategic Reference Frameworks, the National Reform Programmes and the National Sustainable Development Plans. Conclusions are drawn at the level of each objective (Convergence and Regional Competitiveness and Employment) at the EU level.

• Terms of Reference
• Final synthesis report

A study on the Translation of Article 16 of Regulation EC 1083/2006 for Cohesion policy programmes 2007-2013 co-financed by the ERDF and the Cohesion Fund

The study analyses the integration of the gender perspective, non-discrimination and accessibility for disabled persons in Cohesion policy programmes co-financed by ERDF and the Cohesion Fund. 15 examples of good practices across Member States and policy areas will be identified and analysed. Conclusions and useful lessons to strengthen the gender, non-discrimination and disability dimensions in the current and next policy programming period will be derived.

Main findings

• The evaluation found a good overall awareness of the Article 16 requirements in programmes supported by the ERDF (explicit reference in 64% of the programmes analysed)
• In most cases (70%), Member States consider equal opportunities as horizontal priorities and do not devote attention in separate strategies or priority axes. In 22% of the examined programmes the three themes appear as declarative statement without clear targets, relevant selection criteria or obligations in terms of monitoring. Only 8% of the programmes integrate the three themes in a comprehensive strategy with clear identification of problems and quantified targets.
• Unequal emphasis of the three themes: most attention is devoted to gender equality (70%), followed by non-discrimination. Accessibility for disabled has the weakest emphasis (accessibility requirements are usually defined in national legislation).
• The target groups for "non-discrimination" differ across Member States: in EU12 it's mostly ethnic minority groups, particularly the Roma, while in EU15 it is more about women, migrants and elderly.
• The ERDF and the Cohesion Fund promote equal opportunities in: SME support, public transport infrastructure, social infrastructure (child care facilities). However, other policies that can potentially promote equal opportunities through these funds were identified: (R&D, ICT integration in SMEs; cross-border cooperation through integration of cross border labour markets; tourism and culture services for disabled, health infrastructure, urban regeneration especially in remote areas, etc.)
• Experience built upon previous programming periods has contributed to address better equal opportunities in the current programmes. However, in many cases there is a risk of a formal approach stating the problems without a clear vision of how to adequately deal with them.

Recommendations

• The evaluators recommend Managing Authorities to review their performance in integrating the three themes into their programmes by using this Self-assessment guide, specifically designed for them.
• In order to improve accessibility for disabled, Member States could make accessibility to all venues, infrastructures, transport, technologies and services an explicit compliance requirement for project selection.
• The case studies revealed sufficient involvement of bodies and NGOs working on equal opportunities. Member States are encouraged to empower relevant stakeholders to play even more active role in programme implementation through Technical Assistance budget or specific projects.

Documents

• Executive summary
• Final Report
• Case studies of good practices examples
  
• Terms of Reference
• Inception Report
• Intermediate report - annexes - Indicators

• HERMIN: Analysis of the Impact of Cohesion Policy - a note explaining the Hermin-based simulations (May 2007) - Annexes
• QUEST: The Potential Impact of the Fiscal Transfers under the EU Cohesion Policy Programme (June 2007)
• ECOMOD: Study on the Economic Impacts of Convergence Interventions (2007-2013) (November 2007)
  • Simulation results by country with uniform TFP :
  • Simulation results by country with benchmark TFP :
  • Case study Poland: simulation results with TFP 0.03

• Report
  • Annexes
  • Conclusions
  • Movie (28MB)

• Report
• Country reports :

• Report (November 2006)
• Full report :
  
• Executive Summary :

• Report (October 2006)
• Country reports :

This study evaluates the JASPERS initiative from its inception until the end of June 2011. Its main focus is to establish the impact of JASPERS assistance on the quality and timeliness of the preparation, submission, approval and implementation of major projects, in the countries which joined the European Union in 2004 and 2007. These include Bulgaria, the Czech Republic, Cyprus, Estonia, Hungary, Latvia, Lithuania, Malta, Poland, Romania, Slovakia and Slovenia.

JASPERS was created to increase the capacity of beneficiary countries to make the best use of EU funding. By providing technical assistance to improve the quantity and quality of major projects submitted for approval, it was expected that there would be significant benefits in the regions concerned and in the European Union as a whole.

• Final report
• Terms of Reference (July 2011)
• Inception report ( March 2012) - Annex showing proposed case studies
• First Intermediate Report June 2012
• Second Intermediate Report October 2012

The evaluation scope covers not only ERDF regional programmes but also any national programme implemented in the region as well as, only for Cohesion Countries, relevant Cohesion Fund projects. After more than 20 years of implementation (for the EU15) of this reformed cohesion policy the focus on results, effectiveness and performance has grown significantly in recent years. In this context this evaluation aims to feed the debate about the effectiveness of the policy with empirical evidence on what has actually happened on the ground in cohesion policy over successive programming periods.

• Final report - Quality assessment
• Executive Summary
• Case Studies: Algarve - Andalucia - Aquitaine - Basilicata - Burgenland - Campania - Dytiki Ellada - Galicia - Ireland - Itä-Suomi - Nord-Pas-de-Calais - Nordrhein-Westfalen - Norte - North East England - Sachsen-Anhalt
• Terms of Reference
• Inception report

• Ex post evaluation of projects co-financed by ERDF and Cohesion Fund in the period 1994-1999
  
  • Final report - Quality assessment
  • Executive Summary
  The objective of this evaluation is to analyse the long term contribution of 10 selected projects to economic development and as well as the quality of life and well-being of society. The selected projects are major investments in transport and environment infrastructure with a total value over EUR 50 million per project and were implemented during the 1994-1999 programming period.
  The Port of Gioia Tauro - M1 Motorway - Water supply in Palermo - Dublin Waste Water Treatment - Egnatia Motorway - Madrid Metro - Mediterranean Corridor - Solid Waste Management in Northern Lisbon - Environmental Regeneration of Ria de Vigo - Urban Solid Waste Management in Galicia
  First Interim report: Vol.1 Conceptual Framework; Vol.2 Selection of projects for case studies :
  Inception report :
  Terms of Reference :

This study examined two cases of enterprise support – an investment grant available throughout Italy (Law 488) and various SME schemes in the region of Piemonte.
A particular advantage of the study was the access to several detailed administrative data sources from the Statistical Archive of Active Enterprises (ASIA) assembled by the Italian Statistical Agency (ISTAT). This, along with the availability of a clear control group in both cases (narrowly rejected enterprises for Law 488, almost random allocation between different forms of support in Piemonte) gives a high degree of confidence in the results.
In addition, a beneficiary survey was conducted to allow a comparison with more traditional methods of estimating impact.

Results

• Both monitoring data (reporting 82,000 jobs supported) and the beneficiary survey (reporting 36,000 jobs created) proved to be very poor estimators of actual jobs created under Law 488 (12,000). This adds weight to the argument to use counterfactuals (not monitoring or beneficiary surveys) to assess impact.
• Impacts under Law 488 are confined to SMEs – large enterprises are using the money for projects they would have carried out anyhow. Interestingly, this is where the size effect ends – results are remarkably consistent and positive over the subgroups of micro, small and medium-sized enterprises.
• Even allowing for firm size, smaller grants are much more cost-effective than larger ones: cost per jobs averaged €79,000 for the smallest grants (less than €125,000), rising to €489,000 for the largest grants (above €500,000).
• The study provides a first indication of the relative merits of soft loans and grants: the loans had a cost per job around half that of grants plus a surprisingly high impact on investment – EUR 5 per euro of gross grant equivalent.
• The quality of the jobs created (using productivity and payroll costs as a proxy) is usually similar to average jobs in the enterprises concerned. And, in the case of the loans, the quality is actually slightly higher than average.

Final Report

Interim report

This study examined publicly available beneficiary and commercial databases in various Member States. The goal was to see where data allowed a counterfactual impact evaluation of innovation support.
Of 10 cases examined, 5 were eliminated from the analysis because of small numbers of ERDF innovation projects and hence small sample size (Poland, Slovakia, Slovenia, Flanders, London and Wales). Spain was eliminated because of poor beneficiary data, France was eliminated because, although beneficiary data was otherwise complete, it was not possible to tell the year in which the support had been given.
A counterfactual analysis was therefore conducted for the Czech republic (impact on patents) and Germany (impact on R&D investment and a wide range of innovative behaviours, as measured by the Community Innovation Survey).
The study noted a range of problems encountered in the beneficiary data. It therefore made a series of specific recommendations for reporting in the future programming period, including a clear and unique firm identification as well as a description of assistance which makes clear both purpose and timing.

Results

• During the financial crisis, patent applications fell by 63% in non-supported enterprises in the Czech republic but only 14% in supported enterprises.
• The subsidized firms in the German sample show a median R&D intensity of 6.2%. Without a grant from the ERDF, they would have only achieved an estimated R&D intensity of 4.4% - an increase of just over 40%.
• In fact, the representative firm would have had R&D expenditure of 213,000 EUR without ERDF. The grant increases R&D expenditure to about 300,000 EUR. Thus, the treatment effect in terms of EUR amounts to 87,000 EUR, on average, for a typical grant size of up to 51,000 EUR. Although there is a margin of uncertainty here, this suggests that each euro of public money is additional and levers in more than half a euro of extra private money.
• In addition, R&D grants in Germany impacted on a wide range of measures from the Community Innovation Survey, from product and process innovation to initiating new innovative projects (and not abandoning old ones).
• This effect was mostly additive when considering national grants, however there was only a weak correlation with grant size (suggesting smaller grants, repeated in later years if necessary, would be particularly effective – further research would be necessary to validate this finding).

Final report