IMPORTANT LEGAL NOTICE - The information on this site is subject to a disclaimer and a copyright notice.
Brussels, 20 July 2000
"Framework Programme is not enough"
New evaluation report on EU research programmes issued
Keywords: Framework Programme, evaluation
If the European Union wants to face the challenges of the new economy and attain the goals outlined by the Heads of Government in Lisbon, the Framework Programme alone is not enough and needs to be complemented by other tools, states Mr Joan Majó, chairman of the expert panel that has been assessing the EU research programmes carried out between 1995 and 1999. Research Commissioner Philippe Busquin said "when I became commissioner last year I quickly realised that the Community research programmes alone will not be enough to meet the challenges faced by European research. This is why I proposed a strategy for a genuine European Research Area, which has been endorsed by the Lisbon European Council and the European Parliament. In my view, the EUs research activities should be focused on research fields with high European-added value, be based on new forms of management and more responsive and user-friendly procedures."
The evaluation panel was composed of 11 high-level independent experts from 11 countries. It assessed the implementation and achievements of Community research and technological development (RTD) programmes over the period 1995-1999, and covers activities relating to the Third (1990-94), Fourth (1994-98) and Fifth (1998-2002) Framework Programmes.
The panel recommendations are very much in line with the vision of Lisbon. It makes a clear plea to European leaders to make sure that RTD policies in the Member States reinforce rather than duplicate each other. Along these lines, the panel recommends (see executive summary in annex I; the full report is available on http://cordis.europa.eu/fp5/5yr_reports.htm):
- Adopting a European RTD strategy "at the highest political level", then delegate the "task of formulating and implementing this strategy to the European Commission, supported by an appropriate advisory structure".
- Providing more support to Central and Eastern European countries for their RTD activities and taking steps to ensure that the transition to market economy is successful.
- Restructuring and expanding the Framework Programme" and making use of all the possibilities allowed by the Treaty such as "variable geometry" under article 168.
- Encouraging participants to propose "riskier" projects, with a higher probability of failure, but also a higher pay-off if successful.
- Undertaking a "major review" of the management and administration of the Framework Programme through "re-engineering existing structures and procedures, to delegate responsibility for tasks downward within the Commission, or to externalise them" and to "make procedures much simpler and easier to understand".
- Urgently tackling foreseeable "skill shortages" facing Europe over the next decade (combat brain drain, get young people interested in scientific careers, women in science, etc).
As part of the assessment, the panel sent a questionnaire to participants in Framework Programmes. From the 2275 responses received, the following results emerged (see annex II):
- 70% of participants said that the benefits of participation outweighed the costs;
- 71% would not have undertaken the work in the absence of the Framework Programmes;
- 75% of industrial participants said that participation had improved their competitive position.
- 65 % thought the whole application process was too slow and/or costly
- 35 % felt the accompanying documentation was inadequate or difficult to understand.
For further information:
Gilbert Fayl, Head, Programme Evaluation, Research DG
Telephone: + 32-2-295.77.00, fax: + 32-2-296.20.06
Michel Claessens, Communication Unit, Research DG
Telephone: + 32-2-295.99.71, fax: + 32-2-295.82.20
For more information on the European Commissions Research DG, including previous press releases, visit our Web site at http://ec.europa.eu/research/
Executive Summary and Recommendations
The EU currently faces great challenges. At the March 2000 meeting of the European Council in Lisbon, the Union set itself the goal of becoming the most competitive and dynamic knowledge-based economy in the world. This puts research and technological development (RTD) policy at the heart of development strategies.
The most important conclusion of our Panel is that the Framework Programme alone will not be enough to serve the goals set at Lisbon. Although there is much to commend in past and current Framework Programmes, the challenges we face as we move towards the new economy call not only for the Framework Programme itself to become a much more flexible policy instrument, but also for additional instruments and actions.
RTD policy is inextricably linked with policies in other spheres, especially education and innovation. Our Panel is convinced that the required changes need to be conceived within an overall strategy for Europe, articulated at the level of the EU and supported by all the Member States.
The Framework Programme has helped academic and industrial partners all across the EU to tackle problems collaboratively. It has also contributed to the training of researchers and to the development of the European research infrastructure.
There is still a need for these activities and they deserve to be continued, but the scope of the Framework Programme should also be increased in line with the need to meet the Lisbon goals and the demands of enlargement. The Panel recommends:
Maintaining the emphasis on social relevance and continuing to use Key Actions as a way of focusing programmes;
Maintaining a strong emphasis on collaborative RTD projects supplemented by a variety of other actions;
Emphasising excellence and the participation of leading-edge researchers;
Encouraging participants to propose riskier projects;
Enhancing measures encouraging the mobility of researchers within the EU and between the EU and elsewhere;
Retaining support for generic, competence-building RTD activities;
Increasing the emphasis on the research needed to support other EU policies.
But more is needed in Europe today. The EU Treaty allows for the use of other policy instruments to support scientific and technological activity. Some of these have been used, but the Panel is convinced that existing policy tools need to be further exploited in a restructured and expanded Framework Programme.
This under-utilisation is a consequence of the way the Framework Programme is determined and implemented. The Panel recommends a major review of the systems and procedures used to decide overall goals, specify delivery mechanisms and implement programmes. Specifically, we are convinced of the need to distinguish carefully between these activities and to allocate responsibility for them accordingly. We recommend adoption of a European RTD strategy at the highest political levels. The Heads of Government should then delegate the task of formulating and implementing this strategy to the European Commission, supported by an appropriate advisory structure.
These changes will necessitate a greater level of trust by the Member States in the ability of the Commission to deliver an effective Framework Programme. In particular, the Panel sees no need to continue the Programme Committees.
At the level of implementation, a review of the management and administration of the Framework Programme should concentrate on ways of re-engineering existing structures and procedures to delegate responsibility for tasks downwards within the Commission, or externalise them. At present there is excessive focus on adherence to procedures and not enough emphasis on ensuring overall goal attainment.
The changes recommended by the Panel to create a more flexible, expanded Framework Programme will require support at the highest political levels. This is because they call into question some of the basic principles governing the operation of the European Commission as a whole, not just the way research policy is formulated and implemented.
It will be necessary for Heads of Government to reconsider the priority attached to science, technology and innovation. These activities are critical to the development of the knowledge-based society envisaged at Lisbon. Accordingly, the Panel recommends increasing the relative size of the budgets allocated to science and technology compared to other policy domains.
The Panel is convinced that the percentage of GDP spent in the EU on public and private RTD should rise to at least 3% over the next ten years. Higher levels will be necessary without parallel efforts to avoid duplication of effort across the EU. Private sector RTD expenditure will need to be stimulated if Europe is to keep pace with its competitors. The Panel recommends the use of indirect measures such as RTD tax incentives across the EU in order to flag to the rest of the world that Europe is an attractive place to conduct RTD.
RTD policies in the Member States need to reinforce rather than duplicate each other. In the Panels view, the European Commission has a key facilitation role to play in this area. The Commission should take the lead in outlining the steps needed to pool infrastructure and policy-intelligence resources across the EU. The Panel also urges all the Member States to lend their unequivocal support to these efforts.
The enlargement of the EU presents great social and economic opportunities for all, but only if appropriate actions are taken. The Central and Eastern European countries are waiting for the EU to take the lead by implementing a European RTD strategy which takes their needs fully into account. The Panel recommends that support provided to these countries for RTD activities be channelled temporarily through the existing scientific Academies until new competitive structures for the organisation of science and industry can be developed.
The Panel recommends urgent action to counter envisaged skill shortages over the next decade. This will involve measures to increase the attractiveness to young people of careers in science, actions encouraging retraining, and steps to ensure that the potential increase of scientific talent as a result of enlargement is fully tapped. The Panel also supports the creation of truly European centres of teaching and research excellence capable of attracting the best minds in the world to live and work in the EU.
Innovation is another policy area in which new initiatives are needed to improve the position of Europe. Innovation policy is linked with RTD policy but is much broader, involving financial, market, legal, fiscal and cultural aspects, and will require actions outside the Framework Programme. The Panel supports such endeavours and urges the Commission to ensure that innovation-related activities are high on the agenda of actions supported by the Community Structural Funds and the Accession Funds for the applicant countries.
The Panels positive assessment of activities over the last five years is the basis for recommending continuation and expansion of the Framework Programme. The emphasis on collaborative RTD projects was much appreciated by academic and industrial participants, allowing them to undertake strategically important work which would have been difficult to undertake otherwise. Networking, training-related activities and adequate procedures for the involvement of SMEs were also widely regarded as successful features of the Framework Programme.
Concerning programme administration, many participants were dissatisfied with application procedures and, to a lesser extent, with payment delays. The Panel recommends making procedures much simpler and easier to understand.
The overall orientation of the Fifth Framework Programme was endorsed by the Panel, though the initial implementation of the programme was not smooth. The new matrix management structures put in place to ensure adequate communication within and across programme areas did not function well. The Panel recommends an urgent re-engineering of the overall management and administration of the Framework Programme.
The system of evaluation can be considered as well established. Impact assessment should become one of the most important elements of evaluation.
Questionnaire responses were received from 2275 participants in the Third and Fourth Framework Programmes. Fifty-seven percent of the responses came from universities and research organisations; 38% from industry; and 5% from other organisations.
said that the benefits of participation outweighed the costs (72% of academic and 62% of industrial partners)
said that the work conducted was of high strategic importance to their organisations
said that their projects were highly relevant to the RTD goals of the EU
felt that their projects were highly relevant to world scientific and technological developments generally
would not have undertaken the work in the absence of the Framework Programmes
would have done the work but with reduced funds, lower objectives, fewer partners and over longer time-scales
Motives and Goals
said enhancement of their existing knowledge base was a very important goal
said the development or improvement of new tools was a very important goal
said access to complementary expertise was a very important goal
said the formation of new European research partnerships and networks was a very important goal
said access to additional funds was a very important goal
of industry participants said the development or improvement of new tools was a very important goal
of industry participants said that the production of demonstrators, prototypes etc. was a very important goal
of industry participants said that the development of new products was an important goal
of industry participants said that the development of new or improved processes was a very important goal
of industry participants said that the development of new or improved services was a very important goal
Nature of Work
of participants classified their work as applied research (72% of academic and 62% of industrial partners)
classified it as basic research (47% of academic and 17% of industrial partners)
described it as long-term (62% of academic and 55% of industrial partners)
felt that participation had improved their scientific and technological standing (high impact for 55%)
were very satisfied with the quality of their project outputs, 69% with their utility and 60% with their timeliness
attained goals or exceeded expectations in terms of enhancing existing knowledge bases
attained goals or exceeded expectations in terms of the development or improvement of new tools
attained goals or exceeded expectations in terms of accessing complementary expertise
attained goals or exceeded expectations in terms of forming new European partnerships and networks
of industry participants continued to develop project outputs in their R&D units
of industry participants continued further development in their business units
of industry participants said that participation had improved their competitive position (high impact for 30%)
of industry participants had already received significant commercial returns
of industry participants had already received some commercial returns
of industry participants said they had plans for future commercialisation
Programme Management and Administration
thought the whole application process was too slow and/or costly
of respondents thought that application procedures were difficult to follow
felt the accompanying documentation was inadequate or too complex to understand
were dissatisfied with payment procedures (31% of SMEs)
felt that the competence of Commission officials had been an important barrier to the success of their projects
were positive about the help they had received from Commission officials over the lifetime of their projects
PRESS RELEASES | 30.08.2000