Discussions

What's the matter with FET-Open?

Walter Van de Velde

27 October 2015 - updated 4 years ago
26 comments
Total votes: 8

Is FET-Open victim of its own success?

FET-Open is an iconic scheme for FET. Being able to submit at any time a proposal for testing a bright idea for a radically new technological possibility is attractive. Indeed, it is so attractive that the probability to be funded has dramatically dropped because of the very high number of submissions. This is a problem that needs to be addressed if FET-Open is to remain credible as early detector of future technologies.

It is a fact that many proposals in FET-Open do not have a chance to make it pass the FET-gatekeepers (see FET Work Programme), and probably should not have been submitted in the first place. But it is also clear that there are many excellent proposals that do not receive funding, simply because there is not enough money. It leaves a lot of great interdisciplinary thinking unrewarded and a lot of people disappointed (though, judging from the high number of resubmission, not discouraged). The budget for FET-Open is on the rise (budget in 2016-2017 is increased by 50%, reaching 240M€, compared with 160m€ budget for 2014-2015), but that alone is probably not going to solve the problem.

From your point of view, is this problem real? If so, shall we try to solve it? And how?

Note that the best ideas may be implemented as quickly as 2017 and impact on the last 4 years of FET in H2020.

UPDATE (23/12/2015): from inputs received so far the value and necessity of FET-Open is repeatedly stated. It is also stated that the problem of oversubscription is, if not caused by, at least amplified by external factors like that most programmes (European and national ones) are perceived to be too closed, too prescriptive and too down-stream for really new ideas and for newcommers (that often have the new ideas). A radical increase of the FET-Open budget is called for by most (it will probably grow to around 200MEuro per year for 2018, 2019 and 2020) and you all want the focus on breakthrough with impact to be kept, possibly with smaller grants. The call text should be clear and simple (are the current gatekeepers the right ones?), making sure that support servives, for instance from National Contact Points, are fully aware of what is expected. The quality of evaluators must be a priority and the evaluation process must be as transparent as possible (up to radically Open Access) and beyond doubt of fairness. Several contributions points at some form of filtering (before or after submission) or resubmission regulation (very strict, or at least with a clear message whether sensible to resubmit or not), but no consensus emerges on what it should be. However, the use of 2-stage submission is not favoured.

Keep on giving us your ideas on this. In the short term we may implement already for 2017 a combination of measures to reduce the indicated grant size (for instance up to 3MEuro instead of the current 4MEuro), and to reduce the number of cut-off dates to basically one per year, which should increase available funds per cut-off date and decrease the opportunities for resubmission. For 2018 and beyond we will consider more radical solutions as well, in the spirit of FET-Open.

Tags:

Comments

Jarno MA Tanskanen,

15/04/2016 13:22

FET Open is a fantastic funding instrument, probably crucial to Europe, regardless of its budget and of the probability of a project to receive funding. I also find the principles of FET Open excellent and the system just.

A substantial increase in the FET Open budget could most probably be highly justifiable from the points of view of enhancing European competitiveness, wellbeing and development in general. In any case, the problem of oversubscription is probably very hard to overcome and something we probably just have live with. But, it would be valuable to seek methods to decrease the usage of resources for both funding proposal preparation and evaluation.

Many projects are hopefully making true valuable contributions, have real impact, but still it could be worth investigating possibilities to enforce or support the realization of the impacts of FET Open projects. Even though projects may have real visionary and implementable impacts that could make a difference, the impacts may all too often be diluted by ever changing research projects and personnel, dissolution of research consortia, uncertainty of funding, lack of resources for impact building, other scientific interests, and by the lack of long-term commitment to impact building in general. Even after a successful FET Open project, long-term impact building activity may sometimes not even be possible in the current academic research setting.

To enhance these aspects, two possible approaches have come to mind:

A. Presubmission enquiries

There could be a system for presubmission enquiries for funding applications, alike the presubmission enquiry system which Nature has in place for journal manuscripts. By submitting a concise presubmission enquiry, a potential applicant could fast (in a couple of days) get a nonbinding opinion from the Commission regarding if the intended application was in fact truly FET Open material or not. Naturally, regardless of the presubmission enquiry results, the application could be submitted for normal evaluation. Such a system could bring down the number of applications which are not truly FET Open, thus saving the resources of both the applicants and the Commission. A similar system could be put in place also for more topical calls.

B. FET Impact Driver Funding (FET IDF)

FET IDF would be new funding instrument which would allow the main actor(s) of a previously finished project to sustain impact building for a prolonged period post project. FET IDF might work, for example, like:
1. A FET Open consortium or individual partner(s), including an SME, could apply for post project FET IDF any time during the project. Alternatively, instead of the beneficiaries applying for FET IDF, the reviewers, based on a project review, could recommend FET IDF to be granted to a consortium or to certain partner(s) with clearly evident impact potential.
2. In the Final Project Review, a partner or a number of partners of a most excellently executed project with excellent results and clearly identifiable exploitation and impact building pathways, and which had previously applied for FET IDF or for whom FET IDF was recommended by the reviewers, would be granted FET IDF without application.
3. Impact building activity and results should be reviewed every 1-2 years, and if seen excellent (and sufficiently coherent with the original project), be granted for the next 1-2 years without application. The impact building expectations should grow from one period to the next, while allowing the true realization of impacts to take several periods.
4. The maximum term of continuous FET IDF could be, e.g., 10-20 years.
5. FET IDF grant per consortium or partner could be smallish (e.g., for approx. 1 person (Impact Leader, who should solely commit to this task) + some travel, dissemination, and IPR (e.g., novelty assessments) and other related expenses), i.e., just to keep the long-term impact building going on.

FET IDF could allow an actor to work for a prolonged time on the realization of the impacts by, e.g., initiating and steering new projects, steering and enabling R&D, building IPR, productization, and commercialization or public deployment.

Adar Oni-Grinberg,

27/03/2016 12:57

In general, it makes no sense to ask so many teams to put such a large effort when the chances are 2-4% or even lower. Even less than a senior ERC. There are only two ways out:
increase substantially the budget. At least by a factor of 4-5, with success rates of 15% or more. If not, then do pre-proposals or focus on specific themes in order to minimize the amount of time and energy spent and wasted by too many people.

Peter Lemke,

16/03/2016 17:51

First, it is of course funny that proposals are not handled anonymous. This helps to abuse the program for indirect business promotion. "Open" is thus nothing except perhaps the occasional, randomized selection of a few alibi proposals keeping up the appearance to encourage newcomers to the FET program. For how many excellent proposals did the EU - penetrated by lobbyists - probably already acted as a gravedigger?

It would be desirable if rejected proposals would be summarized in a database - some kind of an "idea pool" - which industry has access to. To be classified by the FET "experts" as ineligible, is not such a great feat, since the identification of real potential seems to be a gift some "experts" have a lack of. In the industry the interest in real - then also industry-specific - ideas that lie dormant in an infinite number of rejected proposals might be far greater and this database could help to generate venture capital where the EU as usual fails.

Paul Kidd,

15/01/2016 14:01

Some recommendations were recently made about the future of FET schemes by the FET Advisory Committee. They make interesting reading and can be seen as being more than incremental – they are revolutionary.

Speaking as one supportive of FET – it may in fact be the only thing in H2020 worth funding – and also as one who has been involved in many FET evaluations, in Proactive and Open, both as expert and as recorder, this is what I would say:

The answer to this question is that FET-Open has been expanded in terms of topic coverage and transferred to REA, where it most definitely does not belong, without, it would seem, proper planning and consideration of what FET-Open in expanded form needs (which is more than just a bigger budget). And what FET-Open needs is to be set-free from the tyranny of the European Commission’s antiquated and mechanistic proposal evaluation system, the tyranny of peer review, and REA’s bureaucratic efficiency mind-set. And this requires that FET-Open be taken away from REA and be given a home of its own (something like the ERC), where a 21st century way of carrying out evaluations can be developed and implemented, that is focused on obtaining quality individual assessments, and mostly certainly, does away with all this nonsense that there has to be a consensus, in whatever form that may be (e.g. at a Panel). It also needs something that could be called FET-Rescue, a kind of skunk works that can take forward highly promising visionary proposals that will never be funded, whatever the proposers may do. And it is also the case that FET Proactive now needs a new home, side-by-side with FET Open. The two belong two together, and the artificial separation that has taken place is counter-productive and looks very much to be motivated by bureaucratic notions of efficiency (i.e. cost) rather than what is necessary, strategically.

Athanasios Dimoulas,

24/12/2015 11:56

An obvious way to increase success rate is to reduce the reommended budget per project to about 2 ME with a recommended number of partners ranging beteen 3 and 5. This should be accompanied with relaxed requirements for interdisciplinarity.
Another way is to put restrictions (or penalties) on submission similar to ERC. So the Calls could be more than one in a year, but the projects which do not pass a certain threshold should not be permitted to submit in the next Call the same or similar proposal with the same consortium synthesis. This will discourage people from submitting in any Call without having established a strong case and will reduce the number of submissions. This strategy will not only benefit the success rate but it will also result in higher quality evaluation procedures since less submitted proposals is generally easier to handle.
Finally, I am inclined to favor the 2-stage submission process. Although more combersome and time consuming, the 2-stage submission ensures that the best ideas pass in the first stage and that the scientific excellence that is the primary target of FET Open receives good attention in this first stage. In the second stage, emphasis should be given on impact and implementation

jun-ichi yano,

06/12/2015 16:18

Much is said already, thus what I intend here is to synthesize them
and try to present a critical point of view from there.
I also try to see the issues form a perspective of the whole EU
science funding policies, as my own experience with FET-OPEN is
limited as well as because such a global view is a key here.

Due to the EU austerity policy, science budget would never increase
dramatically for while. This is the basic fact we have to accept.
Thus, instead of wishing that research funding would be increased
somehow, we have to look for a more efficient and fair manner for
exploiting the limited European science resources together.

Simple reflections as presented below would point us that the real
key issue is lack of openness and transparency in process of developing
EU science policies as well as the grant selection processes. In
short, all are currently performed in closed doors.

I would like to much welcome this open forum discussion over internet
from this very perspective.
I would strongly urge that this method is more extensively exploited,
especially, in developing the scientific polities, and defining the research priorities in various domains.

The real issue with the EU science budget is, as explicitly stated by
Stephen Katzenbeisser (02/12/2015), mostly going to "narrowly defined"
specialized themes, to which only limited people can apply by. Thus,
there is no much freedom of choice left for most of the researchers. A
key question is: who are deciding those "narrowly defined" specialized
themes? Clearly the decisions are made behind a closed door without
any substantial inputs from our side.

[EU science policy is, of course, not totally behind the closed
doors, but they do ask comments to a general public (targeting to the
scientists). In fact, I did contribute a comment in a context of
Horizon 2020, but most importantly, I did not receive any feedback
what-so-ever afterwards]

Quite few comments question the fairness of the review
process. However, a more serious issue is the fact that it is hard to
debate on this issue objectively, because these reviews are not
publicly available to all. Again these decisions are made behind the
closed doors.

As the absolute minimum, all the applicant should be given a last chance to
comment on objectivity of the evaluation reviews to their proposals
before a final selection is made (as customarily the case with UK NERC
selection process). Of course, the selection committee is
going to take into account of the rebuttal of the applicant in order
to judge the fairness of the reviews. Accordingly, the marking by a
reviewer must also be adjusted accordingly.

Will it improve a fairness of the evaluation process? We would hardly
know, because the final decision will still be made behind the closed
door.
In this very respect, all the documents associated with the evaluation
process (including a record of the procedure for the final selection)
must be obligatorily put into public domain so that everyone can access
and judge "objectively" the fairness of the selection process.

This would be already a fair amount of progress, because most people
would try to behave better when it is observed in public than
otherwise.

However, this would still not be a final solution: we may still find
the selection process is not fair. Importnatly, now, we can prove it
objectively based on publicly available documents. However, note that
there is no much option to act upon this unfairness.
I know that EU has already set up a procedure for us to set a
complain for a proposal evaluation process [I discussed about
sending this complain to ERC with co-proposers, but we did not].
However, pessimistically, we would just receive a highly-bureaucratic
reply saying "thank you for your complain, we will consider this issue
in next selection procedure", and then nothing would happen.

Thus, we have to think harder how to achieve fairness, openness, and
transparency can be really achieved.

Here, let me change the question for now: what we really need for
European research leading to economical development, or is this too strong
emphasis to the economic development actually damaging the EU scientific research?

In this context, officially FET-OPEN states that its purpose is to
look for innovative, ground breaking ideas [though the comment by Wim
Melis (04/11/2015) questions whether this is practically true)]rather
than simple "continuation" of existing research.

The notions of "innovative" and "ground breaking" somehow assume that
there are quite few genius in EU who will propose those things out of
our surprise, beyond our simple continuation of research.
However, we should definitely avoid such a dichotomy between
"innovation", "ground break" on one hand, and the simple continuation
of existing research on the other hand.

If one knows the history of science little, there were never
any "ground breaking" discoveries out of void totally independent of
continuing research by many scientists in history.
Just consider scientific revolutions achieved by Newton, Darwin,
Einstein.
If one examines their achievements bit carefully, one would realize
that their main efforts consists of synthesizing all the knowledge they
had gotten at that moment, and tries to fill a hole, that could have
been seen by anyone who made the same efforts.
Arguably, their ingenuity resides on finding a way filling up this
hole. However, we have to realize that this is only the last step,
only after a long process of tedious efforts, more like a simple continuation
of existing research.

In other words, there is no "innovation" out of air, but those things
come out only from tedious continuous efforts of existing research.
We should NOT expect that the "brilliant ideas" would come out nowhere,
but these solid ideas come only out of the continuous existing efforts.

However, of course, we must be bit careful here: the "continuation of
existing research" actually has two major different meanings.
What I meant above is a continuing effort step forwards: there must be
an incremental, and critical progress here. Most importantly, one must
try to solve an important problem, how technical, and tiny detail it
may be.
Another meaning is a negative one: a simple repetition of existing
research, trying simple variations of existing work, but those do not
lead to any real progress in critical sense.

I do not say, it is easy to distinguish between these two types of
"continuation".
The key point here is a need to develop a good devise to objectively
measure the existing continuation of research from this perspective.
Again, a certain type of continuing open-discussion forum would help
for this purpose.

We, researchers, tend to try to emphasize the importance of the basic
research against the applications, whereas the current EU schemes tend
to emphasize the applications (and economical impacts) against the
basic research.
However, here, again we should definitely avoid a dichotomy between the
basic and application.

For example, in weather and climate research, the practical goal is to
provide more accurate weather forecasts as well as climate change
projections.
Numerical models already exist for running for these purposes.
However, these are not yet perfect, thus they must still be improved.

Effort for model improvements is not simply matters of applications,
but it consists of various levels, and it is hard to say where the application finishes and basic research begins.
However, more importantly, under this model improvement effort, the required basic research is "naturally" identified.
Such a good basic research would have an almost immediate impact to
applications.
[In this respect, I do not agree with the notion that basic research
is a kind of pool waiting for applications]

For example, in weather and climate modelling, the physical treatment
of clouds is one of the key issues still to be fully resolved. In this
core, we use a formulation called "convection
parameteirzaion". Without getting into the details, all the
specialists would know that both "closure" and
"entrainment-detrainemnt" are the key technical issues here.

In this manner, to a good extent, important basic research can be
identified from a perspective of applications.
Here, I do not argue we can identify the unique priorities, but these
priorities (basic research questions) can be objectively listed, and
their importance as well as possible methodologies for addressing them
can be discussed objectively and openly.

Such must be a true process of deciding a scientific policy, as well
as a selection of scientific proposals. As I emphasized already, such
a process must be open and transparent in order to be objective. The
open-discussion forum should be extensively exploited for this purpose.

Considering the matters in this manner, the most brave proposal I can
make for the proposal process of FET-OPEN is to make the whole process
"open-access", not only the selection process but also the drafting
process, and extensively go through a process of "open discussions"
using a forum page set up.

Openness of proposal drafting process is helpful (though this should
not be imposed to be comprosive, many applicants would gradually find
it useful), because proposers can obtain various external (and
objective) critics, and more importantly make it easier to
organize a consortium by receiving the expressions of interests openly.
In other words, open discussion on the value of the proposed project
already beginning in drafting stage.
This openness also makes it much easier to identify two competing
proposals, and to seek a possible merger, even before the submission.

After submissions, the proposal itself as well as the proposal
evaluations remain open to comments by all.
It would be highly desirable that all the comments are properly
responded either by proposers or evaluaters, as directed.

In order to allow sufficient time for open discussion, we should allow
longer time for drafting as well as for evaluation process.
The process would be slower, however this would be the necessary
procedure to ensure a fair, objective selection process (also
including the process of developing scientific ideas) in order to
exploit limited research funding in more efficient manner, and
distribute it to more appropriate people in a fair manner.

Details of such a highly open project development (both drafting and
evaluation) process must be, of course, still be carefully considered.
However, existing open-access internet journals take an
open-discussion phase, and this system could be taken as a prototype.

It is important to note that the open process of project development
would inevitably asks the questions of research priorities in the
given domain, identifies basic research is required for necessary
applications, and even key technical questions in basic research,
where much investments are required.

In this manner, the open process of project development explicitly feeds back to
the science policy development process.

I read various suggestions about the selection procedures, especially
the two-stage selection. However, those proposals appear to make the
selection process unnecessarily more involved without clear guarantee
of benefits. It seems to me that this open evaluation process would
resolve all those issues in a more straightforward manner.

Finally, an illusion of possibilities for interdiscplinary research
must stop.
The current EU emphasis of interdisciplinarity has led to many
superfluous projects: for example, climate scientists and economists
may work together in order to asses the economical impacts of the climate
change, and possible feedback of the former to the latter.
A climate and an economic models are somehow coupled together, and
they obtain some "numbers".
However, such interdisciplinary projects are performed without any
real mutual understanding: economists would have no idea about
reliability of the climate model, and climate scientists would have slightest
idea how the economics actually works.

True scientific research is a process of developing a logic (that
includes numerical model codes, but that is only a part).
Development of a common logic together is possible only if we speak a
same language.

Interdisciplinarity is a hardly workable scheme for research, unless
the two researchers from two different disciplines are both
bilingual.
Currently, there is no system to educate the scientists in this manner.
Without such a workable system, simply calling for interdisciplinarity
is nonproductive, and likely to deteriorate the quality of the EU
scientific research in long term.

The proposal here may sound too ambitious and unrealistic.
However, I strongly believe this is a workable approach, and I am
willing to take part of its development if I am properly supported.

francesco niglia,

04/12/2015 11:59

It would be interesting to merge some similar excellent projects, so to give the opportunity to lower ranked proposal to access funding for carry out (even) a part of the activities.
Maybe we can ask proposers of excellent (but not ranked for funding) projects to accept to be eventually "side" projects of another similar project (in case there is at least one similar project) ranked for funding. The idea is to give a small amount, let's say € 500.000 as a case study. The proposers, than would be asked to produce a reduced workplan. This procedure often happens in some local Country funding instruments.

nika erjavec,

04/12/2015 09:05

Doubling the budget of FET Open certainly is a step in the right direction. The interdisciplinarity of the programme may be exploited by some, who had not found another (narrower) scheme to apply to. Crucially, though it also allows for bold ideas to connect previously distant fields. Bold ideas are risky and they often come from those who have the most to lose: emerging researchers. Unfortunately, for as long as a CV and connections trump the value of ideas, disruptive thinking will not be recognized. One way of changing this follows what mentioned in a previous post: reviewing the proposals anonymously. That is, the EC should award ideas, not candidates. This may have the added benefit of bridging some of the regional brain drain and age/gender gaps that mar scientific funding in the EU.

Barbara Noziere,

02/12/2015 18:27

Based on my experience with other EC programs (Marie Curie Post-doc 1997 + Reintegration Grant 2005 + Marie Curie Excellence Chair 2005) and outside the EU (8 years in the USA, 5 of which as a professor and many on-going collaborations) I can not stress enough the absolute necessity for the EC to foster technological development by programs such as FET, and at the same time the limits of the present program. Technological development is key for Europe to remain at the top in science, especially compared to Countries such as the USA where technological development is everywhere and rewarded.
But the FET program is much too small budgetwise and way too selective to work as an encouragement to technological development and to make a statistically significant contribution at the European scale. I mean, the extremely low success rate suggests that technological development should be limited to (very) few lucky groups in Europe, while it should be encouraged as widely as possible.
Although I do not have any miracle solution (yet) I would suggest that the selection criteria are way, way too selective. For instance, why should a technological development, on top of being innovative, be also multidisciplinary and, and, and... This eliminates many, truly groundbreaking initiatives, which would be highly rewarded elsewhere. Speaking from my own experience, a project previously funded by the US NSF for its groundbreaking character, and highly rated by the ERC Synergy program (overall grade B), did not even make it through the first selection... Perhaps, a part of the problem might be in the evaluation process, which could be improved. Some the reviews we received were quite unprofessional, suggesting that the reviewers were either overloaded with work or not competent for this program.
Again, I do not have all the solutions yet, but fostering technological development (by definition a very costly activity) is an important problem, that needs to be solved.
Sincerely,

Giulio Santori,

02/12/2015 16:11

It is likely I am repeating some of the valid comments above.
Some sparse thoughts after reading the previous comments:

-My impression is that in many cases technologies are proposed which are not really emerging.
Of course, to be funded a technology should have got a sort of validation before funding, like for example a scientific publication showing evidences supporting an early idea or not published experimental confirmations. NOT PATENTS because patents are to protect ideas which in many cases do not have any experimental or practical confirmation.
On the other side of the coin, a really emerging technology runs the risk to miss interdisciplinary criterion. I am sceptical about the extreme interdisciplinary that FET has got, which certainly works for ICT but not for other areas.

-Two stage submission: I think this is an insane idea. I really appreciate the efficiency of 1 stage submission and the length of the proposal required. I do not see any necessity to go back to a 2 stage submission process to increase artificially the success rate.
What I feel excessive is the space dedicated to information on project management which dominate the project length.

-In many cases, Marie Curie funded projects generate research which is valuable for FET.

Summarizing, key components to increase the success rate can be:
1) Clear statement on the call, orienting people to submit cases only when:
i) the technology is genuinely emerging (no combination of proposed technologies in larger systems, no enhanced technologies);
ii) the feasibility of the technology has been already proved but only at an early stage (evidences from fundamental research or early stage applied research evidences);
2) special channels to promote the continuation of good projects already funded by EU in smaller calls (Marie Curie Actions) in FET.

Best wishes,
Giulio Santori

Said Hamdioui,

02/12/2015 13:59

It is a matter of fact that breakthrough (later commercialized) ideas are the ones that can give Europe a unique position in the world and/or strength it position. … It is also a matter of fact that there are very limited EU programs providing support for such ideas…. FET-Open is a one of the few ways to get support for “beyond-standard- ideas…

I don’t see how we can promote scientists in EU come up with these radically new idea and work on them if FET-Open acceptance ratio is negligible……. this could affect the success of the whole FET-Open and its purpose…

Obviously (seriously) increasing the budget is the minimum that can be done…

Also to save time and energy, it is worth considering making the review process in two stages/steps.. only of you pass the first phase (e.g., no more than 5 pages) , you can apply for the second phase… this will save lot of time for applicants and a lot of money for EU...

It seems also that it is hard if not impossible for people without record to make it for FETopen… that means that no space left for truly emerging ideas from young talented people..

Also the scoring system raises questions… e.g., how can we measure the difference between a projects with scores of 4.5 or 4.6?. In my view this needs (a) redefining and refining of the score system, and (b) better training of the evaluators.

Andrea Vitaletti,

02/12/2015 13:32

There are a number of good comments here with practical suggestions that might help to increase the number of funded projects.

However, I think the main goal should be to increase the impact of FET. As a European citizen I have no problem if 97% of the FET Open projects are not funded, but I would like that the majority of the funded projects will have an impact.

Unfortunately, in my opinion, there are not simple metrics to measure a success FET project and I don't think the "ready-to-market" metric is the most appropriate to evaluate a Future (by definition not in the market) Emerging Technology... also a huge number of publications is not necessarily a good proxy. It is more complex than that and we do not have to make it simpler for the rush to show "practical" impacts.

In many cases - may be all the cases - successful FETs should be "incubated" to eventually provide the future technology promised at the beginning or any kind of significant spin-off. In my opinion, this is a necessary action not yet satisfactory implemented. ... SME instruments are designed to be too close to the market to be appropriate for this incubation phase.

The disruptiveness of a FET is radical and usually envision a new world, the disruptiveness of a SME project envision a new market!

Notice that I'm an entrepreneur, but also a researcher :-)

Hope this helps, Andrea

Derek McKenzie,

02/12/2015 12:24

The first point is that the budget needs to be increased. In FP7, the success rate in FET Open was 6%; whilst this was one of the lowest in FP7, it is a target that the current FET open programme is a long way from achieving and currently too many very high quality ideas are un-funded. The second point is that the understanding of the gate-keeping criteria by many consortia that are contemplating a proposal needs to be drastically improved; currently there is a widespread problem within the FET community that doesn't sufficiently understand the criteria correctly. More consortia that have ideas that are not suitable need to draw their own correct conclusion not to bid, and to be doing so early enough in the process to avoid wasted time and effort, and increase ultimate success rates; this means far more effort to make available to the FET community the materials (e.g. more webinars, a dedicated portal for FET, better understanding of FET amongst NCPs) explaining the criteria more effectively. The submission process should also change, retaining a one-stage submission but introducing a two-stage evaluation to streamline evaluation (saving money to put into more materials, resources explaining the gate-keeping criteria). The first stage of evaluation could be a two-page description (B1) and the second stage of evaluation could be the current 15 page proposal (B2). The number of proposals accepted to the second stage of evaluation could be also be governed by the available budget; e.g. budget of 100m could equate to proposals requesting a total of say 500m are accepted to the second stage.

The criteria could also be made tougher to reduce further the number of poor proposals being submitted, but this should only be implemented once there is a better understanding of the current criteria (a phase 2 implementation).

Theodossis Theo...,

02/12/2015 12:03

FET OPEN is not only an attractive scheme for funding but is also the cradle of new technology for the future. As such it should be at the spearhead of European research and should have a priority in funding even with regard to the ideas schemes.
This is because ideas is a personal grant and takes very much into account the researcher profile but with all the wrong criteria like Nature publications etc etc.
And this is where Europe and the world are making a mistake assuming that when a researcher originates from a pedigree lab with Science, Cell or Nature publications they have to be brilliant. Well this is wrong they can be the far opposite but they will still get the money. Europe when looking at the Researcher profile they should look for highly autonomous rising researchers and especially those who manage to emerge from non-pedigree labs, because these are the pedigree group leaders of the European excellence science.
In that sense FET OPEN should also encompass such criteria.
11 out of 670 proposals when about 200 pass the excellence mark is a pretty poor result. And we are talking about ground breaking ideas which will influence the world of tomorrow, or will not if they never get funded. So money from all other research areas should be streamlined to FETOPEN. Europe and the world are making a grave mistake at the moment to nurture "translational" research especially in the life sciences and "demand" more clinical applications, choking basic research. In my opinion this is the wrong turn as when the pool of concepts derived from basic research is dry, there will be nothing left to translate. Basic research should be nurtured and also that one which currently does not seem to have an apparent application, and Europe must resist the current international frenzy for "translation" in order to become a world leader in the future. This should be a strategic choice. And FET OPEN fits the bill exactly. FET OPEN needs to be allocated the funds to be able to accommodate all the excellent projects. And if more come these should be welcome and accommodated too. FET OPEN makes people think for the future and become inventive and creative.
So in summation I propose for the FET OPEN project:
1. To become the #1 funding scheme in EU Research in size of funds and in priority
2. To evaluate scientists the same way as projects: i.e. on their own autonomous progress
3. To set a short term target of accommodating all excellent proposals to the (reasonable) expense of current translational funding schemes
4. To eliminate the human factor from the choice and allocation of reviewers
5. To Establish a "FET OPEN BASIC", open to non-targeted basic research
6. Provide IP protection to the applicants, as their breakthrough ideas will be "aired" to the reviewers. And although confidentiality agreements may be signed it is in human nature to discuss about an exciting project that one has come across.
7. Set up a modular funds allocation (release) pending on the progress of the project and with specific "go/no-go" points set by the applicants, when decisions for further funding should be taken.
8. Make the FET OPEN projects similar to the NIH applications where in near excellent or low priority excellent applications the reviewer can come up with critical comments which can be addressed (or not) and the proposal resubmitted (or not).
9. FET OPEN projects is where the translation should be focused... Set up a follow-up translational funding scheme for the successful FET OPEN projects.
10. Along the lines of point 9, set up an EU driven venture capital scheme where the EU capitalizes on these ideas. This could be either in collaboration with private funds or pure EU.

Stefan Katzenbeisser,

02/12/2015 11:34

There are already a number of very good comments on this page -- I can only second them.

Nevertheless we should also ask a more general question: why are we seeing so many FET Open proposals these days? From my experience FET Open is often the only EU funding opportunity left for European consortia. In my impression, the scope of recent H2020 calls got narrower and narrower (sometimes the description of work already reads like a "condensed" project proposal); at the same time the "average" technology readiness level was increased to a point which academia-driven consortia are unlikely to fulfill. This used to be different a couple of years ago. Today, a project that is not driven by industry and does not exactly fit into one of the narrow thematic H2020 calls has little chances of getting funds outside of FET Open. I guess we are seeing the effects of these decisions -- this calls for a political debate and solution.

I agree that FET Open is a great scheme (I was involved in 3 FET projects, so I know out of first-hand experience) which should definitively be continued with an increased budget.

Tony Gore,

02/12/2015 11:08

FET-Open has similar low rates as the SME scheme. One of the problems with any scheme is that if the barrier to entry is set low e.g. only a 10-15 page proposal, then too many people are tempted to just throw a proposal together, and this floods the system with low quality proposals. However, two stage evaluations take longer and are more potentially more costly to run, although the first stage can be quick and thus there are many fewer proposals to properly evaluate at the second stage.

One of the problems (not just with FET-Open) is that many proposals are mismatched to the TRLs (Technology Readiness Level) of the calls they are submitted to.

I think that maybe some sort of two stage proposal scheme could be useful, but I would make the first stage very short - along the lines of 1 page to describe the idea, 1 page for the impact. For the last page, I would not bother with implementation - at this stage, if the idea is not good, then the quality of the implementation is simply irrelevant. What is necessary is to demonstrate where the idea fits on the TRL scale, and indicate (based on the assumption that the research will be successful) a roadmap for what will happen after the project e.g. how it will drive future research, and/or how it will eventually impact Europe via societal benefits and/or industrial exploitation. This does not need named organisations, but the sort of organisation (academic and/or industrial) that will pick up the results. It may well be that this "roadmap" is to explore a new class of materials, which would be appropriate if the idea was to develop a new type of material.

So these three pages outline

1) What
2) Potential impact of the results
3) Post project "use" of the results

I would not require any form of consortium at this first stage - we want to evaluate the idea and its potential.

By deliberately making it short, the proposer(s) have to get to the point and describe the key idea. Although I have not evaluated FET-Open, I have a lot of other evaluation experience, and generally find that a good proposal can get the key idea over in the first paragraph. Too many proposal concentrate on the problem they are trying to solve and everything that led up to this rather than describing what they want to do.

This sort of first step can be evaluated quickly and, provided people signed a disclaimer that they wouldn't challenge it, evaluators could give a bit of useful feedback.

In some two step schemes, they have a low threshold, so only really poor things go out at this stage, but most people will accept that a not very good score at the idea stage suggests that they need a rethink if they want to have a chance of success. Others have a high threshold, higher than a second stage, so that only really good ones get serious time spent by proposers and evaluators. Both approaches have the merits and problems. It is possible to actually use two thresholds, and classify the first stage into (and I am paraphrasing here) "rejected - don't bother again with this", "has some merits, but not compelling - rethink it and resubmit to stage 1 if you want" and "good idea, go to stage 2". Instead of setting fixed thresholds, it might make sense to use percentage thresholds - the top 10% going through to stage 2, the next 20% allowed to resubmit and the remaining 70% rejected. It may sound harsh, but there is not point encouraging those that are not in the top 30% to even think of going forwards.

Panayiota Poirazi,

02/12/2015 10:55

The FET Open is a truly wonderful funding scheme as it allows a bottom up approach to frontier research, without strict thematic restrictions, low demands for transnational research, no mobility requirements and an overall flexibility to fund the best ideas possible. The high number of submissions indicates that this type of schemes are precisely what the scientific community wants and needs! As such, the budget for the FET (not just the Open) scheme should definitely increase.

In addition to increasing the budget, the following improvements can be made:

1) Decrease the size (and subsequently the maximum budget) of the FET (Open) grants. True collaborations have been shown to emerge among rather small consortia (2-4 partners, e.g. the HFSP grants). Large consortia are more likely to fail to integrate all partners and subsequently to deliver the promised contributions to science or innovation. This reduction in size will also increase the number of projects that can be funded, thus increasing the success rate.
2) Alternatively, create two categories: small (2-4 partners) and large (5-10 partners) FET grants for more flexibility to also allow funding of large projects.
3) Require and/or reward preliminary data (not incremental efforts). This will identify teams that are already committed to work towards the proposed project, which greatly increases the chances of success and highlights a really important project.
4) Allow single Principal Investigation (PI)-driven grants. Currently (to my knowledge) the only possibility for a sole PI grant in H2020 is via the ERC or the Marie Curie actions. The former requires an established tract record (even ERC STG grants are awarded to PIs with an average of 6.4 years from PhD) and the latter requires mobility. These are serious restrictions, especially for young PIs which constitute the largest pool of European Brain Drain. Given the mission of the FET scheme to fund excellent ideas, these should also be allowed to emerge from individuals and not just consortia. These PI-driven grants will be smaller in terms of budget, thus also increasing the success rate.
5) With respect to the application & reviewing process of FET schemes, this can also be simplified to reveal the best scientific proposals. The reduction of the proposal size to 15 (instead of 60) pages is already a great improvement! Additional steps can include:
5.1) The evaluation criteria currently rely heavily on non-science sections (measures to maximize impact, communication of results, management structure, interdisciplinarity etc). While these are of course important, they should not have the same weight as the excellence of the proposed project and its implementation. More emphasis should be placed on the potential for breakthrough research with potential (not necessarily immediate) for high impact. This will also help identify top projects rather than excellently written proposals (sometimes generated with the help of communication experts).
5.2) The rigid WP organization, with milestones and deliverables could be relaxed to reveal great ideas, that otherwise cannot be elaborated enough in 15 pages. The structure of ERC proposals, where excellence and breakthrough nature are more elaborated on than the work plan structure seems more appropriate for FET proposals than the current work-plan/WP template.

Hope these ideas help!

best wishes,

Yiota Poirazi

------------------
Panayiota Poirazi, Ph.D.
Director of Research
Computational Biology Laboratory
Institute of Molecular Biology and Biotechnology (IMBB)
Foundation of Research and Technology-Hellas (FORTH)
Vassilika Vouton, P.O.Box 1385
GR 711 10 Heraklion, Crete, GREECE

Tel: +30 2810 391139
Fax: +30 2810 391101
Εmail: poirazi@imbb.forth.gr
http://www.dendrites.gr

Chair
http://www.fens.org/Outreach/FENS-Kavli-Network-of-Excellence/

Tarik Ozkul,

02/12/2015 10:54

I have applied to FET open multiple times and get rejected. (With different projects) I am an experienced entpreneur and rarely submit a project unless I already have a patent for it. Having patent in the field means that I know the cutting edge technology and surpassed it. (Obviously having patent does not guarantee that there will be a succesful product, but this why we are applying for the seed fund anyway.) What I have noticed over and over again from evaluaters comments that, the evaluaters do not know the subject they are evaluating well. Sometimes I even dought that they know "english" at the level required to evaluate such projects. After reading comments of an applicant and evaluate on this site, my doubts only got stronger. I suggest increasing quality of reviewers. Academicians may not necessarily be the best evaluaters for commercially intended projects.

Chen Sagiv,

02/12/2015 10:25

As a frequent applicant to FET open, I think that the evaluation, though strict, is fair. There are many good ideas out there that should be funded. The budget is limited, competition is high and this problem is inevitable.
I think that a possible solution on the EU side is to re route applications which pass the evaluation threshold to other relevant calls. Sort of giving a second chance to good proposals for which no FET funding is possible.

Sean McCarthy,

24/11/2015 21:17

The only real success rate is the success rate of proposals that were above the threshold. This means that all the ineligible proposals were eliminated.
To date in Horizon 2020 this success rate is over 33% in other parts of Horizon 2020.
Can anyone provide the equivalent figure for FET?

Aymard de Touzalin,

25/11/2015 09:42

For the call closed end of March 2015, the success rate of above threshold eligible proposals for FET-Open is 3.4% - defined as total number of funded proposals / total number of eligible proposals that were above threshold -. In that call 50% of the eligible proposals were above threshold. The total budget available was 38.5M€. The overall success rate of eligible proposals is 1.7% - defined as total number of funded proposals / total number of eligible proposals submitted -.

Pierre Simay,

10/11/2015 19:07

Pre-screening of proposals to reduce the number of ineligible proposals that are not alligned with FET definition/objectives. Proposers (coordinators) could validate/get feedbacks from their NCPs. This is actually the case, but this could be reinforced with a formal approach and coordinated approach between NCPs. This would request more work for the NCPs and a more active role to check proposals and interact with proposers. NCP FET delegates could get funding from EC (coordination action) to set-up a portal/metholodolgy for proposal pre-screening (though confidentiality might be an issue). The difficilty is to do evaluation before evaluation (is that legal?), but at some point, if this programme remains open, someone has to do a pre-check and make sure that the quality of proposals is high. Or Simply, increase the budget ;-)

Wim Melis,

04/11/2015 16:26

Till now, I have been involved in FET-Open as an evaluator and an applicant. The success rates have gone from low to even lower. However, it seems that there are a few problems from my perspective. Firstly, quite a few of the proposals that I got to review should never have been submitted to FET-Open in the first place, they were simple continuations of existing projects, and so not future or emerging and high risk. However, having seen some discussions by the European Commissions earlier today in this context, it seems that these may possibly more likely what they are looking for. If that is the case, then that means there is no space left for truly emerging ideas, but only for incremental improvements "disguised" under another name. Secondly, I would vote for reviewing the proposal anonymously, and so the focus is purely on what the proposal is, as if people can come up with the great idea they are also best positioned to execute it. We need to have more trust in people as such, if you don't trust them, then it kind of goes wrong from the start. Thirdly, from the perspective of my own submissions, and the comments on them, I noticed that many reviewers had a very limited view as to what would be novel and ground-breaking. Most of the comments were in the line of this is not current technology, well no, it should be ground-breaking. This means it should break with existing thinking, take well estimated risks and see where we get with them.
To me this shows some fundamental issues, which require clarification from the commission, probably more budget and a better training of the evaluators on what really is considered ground-breaking, and make it all anonymous, as the concept of "you need to have a good name on board" should not matter here, it is about excellence, and that comes from the idea itself. If Europe wants to "set the standard" with regards to research then they will need to work this one out quickly.

Matteo MASCAGNI,

30/10/2015 11:09

Indeed the FETOpen successful rate is too low and this could affect the success of the whole scheme. Of course the problem might be easy solved by adding more money to FETOpen. Another possible solution is implementing ASAP the "Seal of Excellence" instrument even on FETOpen. At the moment the "Seal of Excellence" is implemented on the "SME Instrument" only but it is not limited to that instrument so it could be extended to others. The proposals that are evaluated as “excellent” under FETOpen but cannot obtain financing from FETOpen, could use the "Seal of Excellence" in order to access Structural Funds that are indeed available, especially in some Member States that are unable to spend all the ESIF funds available. According to Dimitri Corpakis (HoU RTD.B5) the "Seal of Excellence" could be even "an automatic" instrument so the “excellent” proposals should not be evaluated again but they could be funded by using indirect ESIF funds just because they got the "Seal of Excellence". More info here:
http://ec.europa.eu/regional_policy/sources/conferences/etc2015/dimitri_...
http://europa.eu/rapid/press-release_SPEECH-15-5243_en.htm

paul hearn (not verified),

28/10/2015 21:03

raise budget x 10, do not allow resubmissions, do not allow any coordinator submitting a below threshold proposal to compete in the next round.

Jacco Konjin,

28/10/2015 11:31

Budget should certainly be raised. Too many excellent projects go down the drain. So this level of excellence guarantees a very efficient and successful use of extra budget, that should be an incentive to the EC to invest more in Fet-Open.
Proposals with 4.5 and higher are encouraged to apply again. Maybe the divide between promising and not-applicable proposals should be made clearer in the scoring. If the technology is not rated groundbreaking/radical/game changing enough, the score should be lowered quite heavily. We find it difficult to discourage proposers with scores around 4.0. The message whether or not a proposal fits the FET-open principles should be clearer with really high (encouraging), or really low (discouraging) marks.

Digital4Science

ARCHIVED PLATFORM

This website is archived since mid-May 2021 and it is not possible to login anymore.

But don't worry there is a new Futurium platform and all active communities have been migrated.

Discussions

What's the matter with FET-Open?

Comments

ARCHIVED PLATFORM

This website is archived since mid-May 2021 and it is not possible to login anymore.

But don't worry there is a new Futurium platform and all active communities have been migrated.

Share this:

Comments