This study examined two cases of enterprise support – an investment grant available throughout Italy (Law 488) and various SME schemes in the region of Piemonte.
A particular advantage of the study was the access to several detailed administrative data sources from the Statistical Archive of Active Enterprises (ASIA) assembled by the Italian Statistical Agency (ISTAT). This, along with the availability of a clear control group in both cases (narrowly rejected enterprises for Law 488, almost random allocation between different forms of support in Piemonte) gives a high degree of confidence in the results.
In addition, a beneficiary survey was conducted to allow a comparison with more traditional methods of estimating impact.
This study examined publicly available beneficiary and commercial databases in various Member States. The goal was to see where data allowed a counterfactual impact evaluation of innovation support.
Of 10 cases examined, 5 were eliminated from the analysis because of small numbers of ERDF innovation projects and hence small sample size (Poland, Slovakia, Slovenia, Flanders, London and Wales). Spain was eliminated because of poor beneficiary data, France was eliminated because, although beneficiary data was otherwise complete, it was not possible to tell the year in which the support had been given.
A counterfactual analysis was therefore conducted for the Czech republic (impact on patents) and Germany (impact on R&D investment and a wide range of innovative behaviours, as measured by the Community Innovation Survey).
The study noted a range of problems encountered in the beneficiary data. It therefore made a series of specific recommendations for reporting in the future programming period, including a clear and unique firm identification as well as a description of assistance which makes clear both purpose and timing.
This study tracked 480 firms (253 Invest NI clients and 227 non-assisted firms) for which annual GVA, turnover and data were available. The goal was to estimate the impact of assistance using counterfactuals. This was not straightforward, since supported firms can be quite different to non-supported firms (making it more difficult to find similar "matches").
However, the following tentative conclusions were reached:
This study compared enterprises in Eastern Germany which benefited from investment or research grants with similar, but unsupported, enterprises. There were two specific samples: the IAB Betriebspanel for enterprise support and Gefra's survey of enterprise R&D in Thuringia. To ensure robust results, various comparison methods were used (including propensity score matching, controlled difference in difference and instrumental variables).
The second round of the URBAN Community Initiative supported 70 urban neighbourhoods in crisis, facing challenges such as high unemployment, crime, social exclusion and dereliction. The ex post evaluation included an examination of various indicators with a view to counterfactual impact evaluation. A particular focus was a simple difference-in-difference analysis of unemployment rates (a comparison of the change in the area with changes in neighbouring areas or in the parent city).
Unfortunately, the necessary area data were not available in half the neighbourhoods and the results for the small sample remaining were inconclusive. In addition, unemployment was just one target of URBAN II programmes – the tracking of other goals was often more patchy than for unemployment. This underlines the importance of programmes collecting good data on the things they are trying to change.