Evaluation Process and Results

Independent experts evaluate your proposal following this 3-steps process:

• Phase 1 — Individual evaluation
• Phase 2 — Consensus group
• Phase 3 — Panel review

All proposals within a call (or within a coherent part of a call) are evaluated together. See specific cases in the introduction.

Evaluation procedure

Briefing of Experts

Before starting the evaluation process, the experts are briefed on:

• the evaluation processes and procedures (including selection and award criteria)
• the content of the R&I topics under consideration
• the terms of their contract (e.g. confidentiality, impartiality, conflicts of interest, completing tasks and approving reports, penalties for non-compliance)
• disregarding excess pages
• the need to evaluate proposals as they were submitted, rather than their potential should certain changes be made.

• In Horizon 2020, there will be no scope for recommending improvements to proposals (including improvements on the budget). In particular, proposals with a significantly inflated budget, taking into account cost efficiency considerations, will receive a lower score and may not pass the threshold.

Each full proposal is evaluated by at least three experts.

Specific cases:

• For two-stage submission schemes, there is a first-stage and a second-stage evaluation (against the evaluation criteria for each stage). In a two-stage submission scheme, proposals must pass all thresholds to pass to the second stage.
• For calls with multiple cut-off dates (e.g. SME instrument actions phase 1 and 2) there is an evaluation session after each cut-off date and the final closure date (normally within a month). After each cut-off date, the submitted proposals are grouped, reviewed and ranked together. Evaluation results are made available immediately.
• If the work programme/call provides for a combination of submission with multiple cut-off dates and two-stage submission scheme, the first-stage short outline proposals may be after each cut-off date and the full proposal will be evaluated after the intermediate closure date for the second-stage evaluation.
• For ‘multi-step evaluations’, different experts may be examining the different criteria. Proposals failing a threshold score may not progress to the next step.
• To evaluate your capability, the experts will also give an opinion on your operational capacity to implement the action.

For low value grants and for the first stage of two-stage procedures a minimum of two experts may be used in certain cases.

Individual evaluation

Experts work individually. Each expert gives a score for each criterion, with explanatory comments, and prepares an ‘individual evaluation report (IER)’.

They also indicate if the proposal:

• falls entirely outside the scope of the part of the call which they are evaluating or
• involves security issues that will need further scrutiny.

Consensus group

After carrying out an individual evaluation, an expert will join other experts who have evaluated the same proposal in a consensus group, to agree on a common position, including comments and scores.

Each group is assisted by a moderator who:

• seeks a consensus, impartially and
• ensures that each proposal is evaluated fairly, according to the evaluation criteria

The moderator is normally a Commission official.

Specific cases:

• If foreseen in the work programme/call, an arithmetic average (i.e. median or mean value) of the individual scores may be taken as the consensus score (e.g. for the first stage of two-stage submission schemes, SME Instrument actions).

• The ‘mean’ is the total score of the experts, divided by the number of experts.
• The ‘median’ is found by arranging all the scores from lowest value to highest value and picking the middle one (e.g. the median of {3, 5, 9} is 5).
• If there is an even number of experts, then there is no single middle value; the median is then the mean of the two middle scores (e.g. the median of {3, 5, 7, 9} is (5 + 7) / 2 = 6).
• To evaluate your capability, the experts will also give an opinion on your operational capacity to implement the action.

• If foreseen in the work programme/call the consensus report may consist in a collation of the individual evaluation reports or extracts from them (e.g. the first stage of two-stage submission schemes; SME Instrument actions)
• If you have submitted your proposal to the Commission/Agency previously under Horizon 2020 or any other programme in the past two years, and if the work programme topics and criteria were comparable, the moderator of the consensus group may give a copy of the previous Evaluation Summary Report (see below) to the experts.

If a consensus group cannot reach a common view, the consensus report will set out both the majority view and the dissenting views.

• In some cases we may ask additional experts to examine the proposal, to establish whether a clear majority view exists.

Panel review

After the consensus phase a panel of experts:

• reach an agreement on the scores and comments for all proposals within a call, checking consistency across the evaluations
• if necessary, propose a new set of marks or revise comments, and resolve cases where evaluators were unable to agree
• rank the proposals having a qualifying score and give a priority order for proposals with the same score

The panel may comprise experts from consensus groups, new experts, or a combination of the two. There may be one panel covering the whole call or several panels covering different parts of the call. Each panel will be responsible for one or more ranked lists, as defined by the indicative budget and call conditions set out in the work programme.

Proposals with the same score

The panel will recommend a priority order for proposals with the same score, using the procedure given in the work programme.

Hearings

The Commission may arrange hearings as part of the panel review. In this case, invitations are sent to the coordinators of proposals that achieved consensus scores that are above the individual and overall qualifying thresholds and, sometimes, also to those with scores above individual thresholds but below the overall qualifying threshold.

The coordinator may be asked to provide further information to clarify the proposals and help the panel establish its final ranking list and scores.

This information can be provided in writing, by phone or by a remote video interview, provided both parties agree.

The panel meeting minutes (including the panel ranked list) is signed by at least a majority of the experts present at the panel meeting.

The panel report is signed by the panel rapporteur and validated by the panel chair.

Outcome of evaluation

Final evaluation results

Before notifying coordinators of the final evaluation results, the Commission reviews the results of the experts’ evaluation and puts together the final ranking list.

Proposals will not be offered funding if the Commission finds that it is already funding very similar work elsewhere, or a proposal is in any way manifestly contrary to established EU or Euratom policies.

Final rankings

The Commission:

• produces a ranked list of proposals, including all proposals with scores above the qualifying score
• draws up a list of proposals for possible funding from that list. The number of proposals in the list depends on the available budget.

Each page of topic conditions of a call on the Participant Portal gives approximate timelines when coordinators will receive information on the outcome of the evaluation.

Reserve list

If the available budget is too small to fund all proposals that reached the qualifying score in the evaluation round, some proposals may be put on a reserve list – of proposals that may be offered funding if a higher-scoring project does not go ahead or additional funds become available.

When a proposal is placed on the reserve list, the coordinator is informed. The Commission may also specify a date after which grant agreement is unlikely to be offered.

Rejection decisions

The Commission will notify proposal coordinators if their proposal has been rejected because:

• it is found to be inadmissible or ineligible (before or during the evaluation)
• it falls short of the relevant thresholds
• it is too far down the ranked list to qualify for the limited amount of funding available
• if it fails to obtain ethics clearance, following an ethics review (see Article 13.3 - Rules for participation and the ethics section
• it raises security concerns.

Invitation to prepare grant

The coordinators of successful proposals for which funding is available, are invited to start preparations for the grant.

You are informed about evaluation and ranking results by an e-mail notification from the Commission. The same notification will also arrive to My Notifications Inbox on the Participant Portal.
• It announces that the Evaluation Result Letter is available on the Funding & Tenders Portal. To read it, you need to log on with your individual account, go to My Proposals page under My Area section, click the 'Actions' button, then 'Follow-up'.
• If your proposal is successful, you will receive in the same notification both the information that the Evaluation Results are available on the PP and a request to start preparing grant agreement data.
  In this case go to My Projects page under My Area section, click the 'Actions' button, then 'Manage Project'.
• It is also possible, when the invitation to start grant preparation is delayed, that you may still receive two different notifications - first about the Evaluation Results, then another about the start of grant preparation.

To follow the further steps of preparing your grant, please check the next section: Grant preparation.
You will receive e-mail notifications about all the actions that you need to carry out for the grant preparation. For more detail, please see the list of all the grant-related notifications and their recipients.