Discussions

e-Infrastructures WP2016-2017: are you ready for the TRLs?

  • Dimitrios Axiotis profile
    Dimitrios Axiotis
    9 October 2015 - updated 4 years ago
  • 7 comments
    Total votes: 2

As the WP 2016-2017 for e-Infrastructures has been released you may have noticed that the notion of Technology Readiness Levels (TRLs) has been incorporated in the call text.

The call is structured in two Themes:

  • Theme 1 - Integration and consolidation of e-infrastructure platforms supporting European policies and research and education communities
  • Theme 2 - Prototyping innovative e-infrastructure platforms and services for research and education communities, industry and the citizens at large

The call text reads:

Under Theme 1, only platforms and services based on systems and technologies that have achieved Technology Readiness Level 8 (TRL 8) before the start of the project will be supported.

Under Theme 2, only platforms and services based on systems and technologies that have achieved at least TRL 6 before the start of the project will be supported and it is expected that they will have achieved TRL 8 by the end of the project.

Additional evaluation sub-criteria have been set under the Excellence Criterion:

  • the extent to which it is demonstrated that the concerned platforms and services are based on systems and technologies that have reached TRL 8 before the start of the project.
  • the extent to which it is demonstrated that the concerned platforms and services are based on systems and technologies that have reached TRL6 before the start of the project and will be brought to at least TRL 8 by the end of the project.

In the light of the preparation of your proposals, we would like to invite you to share your views on the following topics or any other that you may deem important:

  1. Do you have a clear view on what are the TRLs and what is their impact on your proposals?
  2. Are you going to use a specific methodology to identify and classify your existing platforms/systems and services into TRLs?
  3. Do you think you will be able to provide this information in a clear and unambiguous way that is not open to different interpretations in your proposals?
  4. How do you plan to measure/prove that the platforms and services will be brought to at least TRL 8 at the end of the project?
Tags: 
e-infrastructuresWP2016-2017TRL

Comments

Log in to post comments
Jarkko Siren's picture
Jarkko Siren,
22/10/2015 11:48

Related post on service catalogues: https://ec.europa.eu/futurium/en/content/service-orientation-catalogue-s...

Total votes: 1
Toby Young's picture
Toby Young,
15/10/2015 17:31

Good afternoon , all

We have implemented and operated a Product Management process within GÉANT for many years, and historically this has been adequate in a closed world of GÉANT service development, evolving when new project cycles started (typically every 2-3 years)

The WP16/17 Program introduces welcome opportunities for the e- infrastructures to align not only service portfolios, but also to use a common measurement mechanism to assess and present the service catalogues to the wider community. In doing this we enable and empower our user communities and SMEs to examine, understand and use our combined service sets to their best advantage, a fundamental requirement of the Programme.

Having reviewed the TRLs as presented by the EC and reviewed "Vertical Sector" variants (space research, chemical engineering and microbiology) it seems clear that TRLs need to be made relevant to our environment i.e. we need to adapt existing processes and standards to ensure they will benefit everyone within the R&E environment. THis work is underway, of course, within GEANT, and I think leveraging off of work undertaken on FITSM is a great suggestion. We did look at this in GEANT last year and had plans to adopt this approach in our next iteration

Toby Young
Head of Product Management - GEANT

Total votes: 3
Owen Appleton's picture
Owen Appleton (not verified),
14/10/2015 16:17

I'd like to wholeheartedly support Tiziana in this. The essence of the issue is that we need Service Levels to be described and considered as well as TRLs. There is support to do this, such as through FitSM, but it also requires education and training as understanding and describing service levels is something less familiar to the research communities regularly submitting projects.

To summarise the points below
- We need Service as well as (or least included within) Technology Readiness levels, the best tech delivered unreliably is not valuable.
- Service levels and service orientation require training on a new topic, it is not socialised int he research domain.
- Training was previously supported by the Commission through FedSM, but this is now over, and while training continues, not all research group may choose to purchase it
- FitSM provides free maturity assessment tools and assistance with SLAs that could be very useful given some level fo community training.

In more detail: I was involved with developing FitSM, and also provide training and consultancy on it to members of the e-Infrastructure community. One key reason for the existence of FitSM and the idea of IT Service Management and service levels in the eScience and e-Infrastructure sector was the need for a service-oriented not product-centric view. Many of the technologies used are quite mature but the communities behind them lack experience in Service management and service provisioning such that the ‘overall’ readiness level is not always sufficient even when the TRL is likely high.

The move to a service economy, or to take the cloud paradigm, the proliferation of PaaS and SaaS as well as just IaaS/baremetal, makes this very important and timely.

The question is how to support this in the upcoming Work Programme. Asking for TRLs is a great idea but there should also be a request for Service Level requirements and foreseen service delivery levels. We can ask for Service Levels but while our communities are technically very advanced, many organisations lack the experience in IT Service Management necessary to describe service levels useful, and customer group are unclear what to ask for. We suggest that some basic ITSM education for key groups would be very advisable, and certainly FitSM is a good option here as it offers 1-day foundation courses. Previously many individuals were trained through the FedSM project, which Tiziana referred to and which concluded in August this year, but while training remain available and is continuing, ti is no longer financially supported by the Commission. A way to make low-cost or free-at-the-point-of-use training for the community would be most helpful in this transitional period.

With some agreed conceptual models, through FitSM or the compatible ISO/IEC 20,000 and ITIL, it would be possible to request stated maturity levels (through FitSM part 6 maturity assessment) or even clearly stated Service Level Targets that would later go into SLAs between providers and research customers. Combined with the TRLs this would make a compelling package that ensures projects dressed real and constructive research needs, with a pipeline toward commercial use. With TRLs alone, however useful, there is a risk of great technology delivered in ways that researchers will not find valuable.

Owen Appleton
Member of ITEMO & the FitSM committee
S3IT, University of Zurich, Switzerland.

Total votes: 5
Tiziana Ferrari's picture
Tiziana Ferrari,
13/10/2015 12:35

Thanks Augusto for the information provided. As the proposed TRL levels of any particular system or technology used in platforms and services will be assessed by the evaluators, as you have explained, it would be good to have some agreement on what best practices to adopt, these would help the work of the evaluators when different ations have to be compared, as well as of the partners submitting their proposals.

We have a great wealth of best practices and standards in service management that can be adopted as reference. Among these, the FirSM standard (based on ITIL and ISO 20k) is originated by a project co-funded by the European Commission under the European Union's Seventh Framework Programme for research, under grant agreement no 312851.

Tiziana

Total votes: 4
Augusto Burgueño Arjona's picture
Augusto Burgueñ...,
13/10/2015 08:19

Thank you Tiziana, Davide and Dimitris for having engaged in this discussion. TRL is a key concept in workprogramme 16-17 and it is important that it is well understood. I'm sure that this conversation will help clarify the questions that its introduction may have raised.

The only normative reference to the definion of TRL levels in the context of workprogramme 16-17 is the one provided in the workprogramme itself. Other references exist, but we have not checked if they are compatible.

Also, the workprogramme does not impose any particular methodology to determine the TRL level of systems and technologies. What has to be kept in mind is that the proposed TRL level of any particular system or technology used in platforms and services will be assessed by the evaluators, since it is part of the excellence criteria in the following topics:

○ For EINFRA-21-2017 (b) and EINFRA-22-2016:

The extent to which it is demonstrated that the concerned platforms and services are based on systems and technologies that have reached TRL 6 before the start of the project and will be brought to at least TRL 8 by the end of the project.

○ EINFRA-11-2016 and EINFRA-12-2017:

The extent to which it is demonstrated that the concerned platforms and services are based on systems and technologies that have reached at least TRL 8 before the start of the project.

Finally, it is important to underline that by introducing TRLs in worprogramme 16-17 we will achieve the following objectives:

○ The distinction between Theme 1 (Integration and consolidation of e-infrastructure platforms supporting European policies and research and education communities) and Theme 2 (Prototyping innovative e-infrastructure platforms and services for research and education communities, industry and the citizens at large) is based on a verifiable criterium (at least TRL 8 for Theme 1 and at least TRL 6 for Theme 2)

○ The success of projects in Theme 2 will be judged, in part, on the TRL level reached at the end of the project (hopefully at least TRL 8)

○ Activities addressing systems and technologies of TRL level 5 or below are out of scope in this workprogramme

I hope this helps,

Augusto.

Total votes: 3
Tiziana Ferrari's picture
Tiziana Ferrari,
12/10/2015 11:54

In EGI the methodology to identify and classify existing services into TRLs will be part of the IT Service Management System of EGI (https://wiki.egi.eu/wiki/EGI_IT_Service_Management), which is based on the FitSM standard (http://fitsm.itemo.org/). FitSM was developed in the context of the EC FP7 project FedSM and is specifically geared for federated infrastructures.

Specifically, the TRL will be an attribute of each service, that will be part of the service description template: https://documents.egi.eu/document/2165
The TRL is periodically reviewed and updated according to the status of the service, as part of the existing Service Portfolio Management process (https://wiki.egi.eu/wiki/EGI_Service_Portfolio_Management).

Only services at TRL 8 and 9 will be part of the service catalogue. The catalogue will be public (the current version is being revised to adhere to FitSM).
Services that are part of the catalogue at TRL 8 and 9 will have associated Service Level Agreements and Operational Level Agreements and/or Underpinning Agreements with the service providers, these documents will unambiguously describe the service properties and levels offered, as well periodic service level reports. Service Level Management (https://wiki.egi.eu/wiki/EGI_Service_Level_Management) and Service Reporting management (https://wiki.egi.eu/wiki/EGI_Service_Reporting_Management) are two processes that are required by the FitSM standard.

For services at lower level of readiness, EGI adopted a software provisioning process to assess software maturity (https://wiki.egi.eu/wiki/Software_Provisioning_Process). This process is used for software quality assurance, quality definition, validation, testing and rolling to the production infrastructure. Only software that passes all the tests including the final test in the production infrastructure can be considered to be ready for TRL 8.

We recommend the adoption of FitSM, our case study is described at: http://fitsm.itemo.org/case-study-egieu. FitSM training is available for who is interested: https://www.egi.eu/services/catalogue/fitsm/

We are interested in learning about the experience of other e-infrastructures.
Tiziana Ferrari/EGI.eu Technical Director

Total votes: 2
Davide Salomoni's picture
Davide Salomoni,
12/10/2015 10:57

We have started discussing this point of TRLs within the INDIGO-DataCloud project. I think that more details on what is actually meant esp. for TRL8 would be helpful.

For example: http://ec.europa.eu/research/participants/data/ref/h2020/wp/2014_2015/an... for TRL8 says "system complete and qualified". This should be something above TRL7 ("system prototype demonstration in operational environment") and below TRL9 ("actual system proven in operational environment"). So my understanding is that a TRL8 system or technology is not a demo but is still not proven in operational environments. Concrete examples would probably help here.

Now, if I look at a more verbose definition of TRLs, such as the one issued by NASA (http://www.nasa.gov/pdf/458490main_TRL_Definitions.pdf), there one finds statements like, for TRL8, "Actual system completed and "flight qualified" through test and demonstration". They then distinguish between "hardware" and "software". For software (which is what interests the current INDIGO project the most), they say "All software has been thoroughly debugged and fully integrated with all operational hardware and software systems. All user documentation, training documentation, and maintenance documentation completed. All functionality successfully demonstrated in simulated operational scenarios. Verification and Validation (V&V) completed."

Is this view coincident with the one that the EC has for the WP 2016-2017? If so, can this be made explicit in the EC documentation? If not, what is the difference?

Total votes: 2