Methodology overview

In order to model the semantics for different types of evidence exchanged in cross-border administrative procedures, the methodology envisions the following six phases:

• Phase 1 - Identify and analyse existing standardisation efforts, evidence types and data models
• Phase 2 - Mapping of OOTS requirements with reference models
• Phase 3 - Complete the semantic expression of OOTS requirements
• Phase 4 - Draft and validate the data model
• Phase 5 - Select controlled lists
• Phase 6 - Create distributions and publish documentation

Overall, these phases aim at reaching consensus on the structured data to be used for exchanging evidence types related to the procedures in scope of the OOTS.

This process should be placed in the broader context of the OOTS deployment which includes:

• Evidence mapping - this activity is carried out upstream and its output is a prerequisite for the data models standardisation. The evidence mapping identifies the OOTS requirements and information requirements and validates them with member States.
• Technical Proofs-of-concept - this activity can be conducted in parallel to the data modelling and allows to test the technical interconnection with related systems and test the data models

The involvement of domain experts (preferably from each Member State) in this kind of discussion is key to ensure collaboration between Member States. Their knowledge of the different specific features of national use cases and evidences types will streamline the process of selecting the most relevant evidence types to be modelled.

Confluence is used as the main platform for reviewing the data models in a collaborative manner. To this end, specific 'issue' pages will be progressively created for discussing and reaching consensus on specific topics, in preparation of sub-group meetings, to be organised in video-conference.

Engagement is a key element to the success of the different stages of the methodology. Having a high degree of participation from MS is therefore essential to achieve quality results which are consensus-driven. Sub-group members, i.e. Member States representatives, should therefore be well represented during throughout the process.

Decision tree for performing semantic analyses and defining properties

This methodology, as part of its six phases, includes two crucial activities:

• the process for conducting semantic mappings  (to ensure alignment between the OOTS and a related system), described in Phase 2
  This first activity requires to perform a semantic analysis to verify the conceptual compatibility (semantic alignment) between the the data model of the related system between and the OOTS requirements. It  also allows to define the level of technical compatibility (technical alignment) between the related system and OOTS. This activity is necessary for Cases 1,2 and 3 described in the Introduction.
  
  
• the process for identifying relevant ontologies and use them for defining OOTS information requirements, described in Phase 3
  This purpose of this second activity is to define information requirements choosing a an appropriate vocabulary from existing ontologies. Semantically defining the OOTS information requirements is a necessary step in order to provide the means to exchange structured data in the context of the public procedures under the scope of OOTS. In turn, this makes possible to map the OOTS requirements against the semantic assets used by related systems (e.g. EUCARIS), to enable interconnectivity and interoperability.
  In case it is not possible to identify a vocabulary from existing ontologies to define the OOTS information requirements, a new one will have to be created.
  
  The following diagram visualises the process described above - which is further described in the following paragraphs - by means of a decision tree to be used for guiding the analysis and definition of the Information Requirements:
  
  

Phase 1: Identify and analyse existing standardisation efforts, systems and data models

This phase includes the following steps:

• Step 1 - Gather existing standardisation efforts, systems and data models

• Step 2 - Analyse the main existing standards and/or systems

• Step 3 - Identify additional complementary standardisation efforts (optional)

Step 1 - Gather existing standardisation efforts, systems and data models

Objective

Gather information in order to have a global overview of data models, and/or standardisation rules implemented and used across Europe and leverage this insight to profile a OOTS data model for specific evidence types.

Key activities

• The sub-group members contact national domain experts in order to identify and share existing standardisation efforts (models or policies).
• The responsible DG shares information on existing models, standardisation efforts or policies.
• The sub-group team (Editors and Coordinator) collects information from the sub-group members and the responsible DG.

Description

The sub-group team (Editors and Coordinator) asks to the sub-group members (national coordinators and national domain experts) to share information they possess related to the OOTS data model for specific evidence types being built.

Similarly, the relevant policy DG(s) with competencies in relation to the scope of the evidence being modelled are engaged and asked to share relevant information and existing legal pieces of work (and/or other relevant pieces of work). Whenever possible they will be asked to indicate any preference in relation to the standard/model to be adopted as part of OOTS.

Examples of relevant data models and standardisation efforts at European level are for instance ELM (Education domain) and EUCARIS (Vehicle domain). However several other initiatives - also at national level - might exist and knowledge about their existence might be relevant for assessing the technical feasibility of data standardisation in a given domain. 

Tool(s)
An online collaborative tool (e.g. Confluence) to share relevant documentation with the sub-group

Step 2 - Analyse the main existing standards and/or systems

Objective
Once information on existing standards and/or systems in a given domain has been collected, analyse them in order to identify the one(s) with the highest potential to cover the OOTS requirements previously identified. 

Key activities

• The sub-group team (Editors and Coordinator) analyses the information on existing systems/standards collected in Step 1, and identifies any potential related system or reference model
• The sub-group team (Editors and Coordinator) shares the outcome of the analysis information with the relevant policy DGs and with the sub-group members, for validation

Description

As stated in the OOTS Implementing Act, ‘the OOTS should build on the work already done and exploit synergies with other existing systems for the exchange of evidence or information among authorities […]'. Accordingly, in this phase is necessary to assess whether an existing standard/system in a given domain has the potential to become an OOTS 'related system' and therefore be reused in order to enable the exchange of structured evidence types.

In this context, for ‘relevant’ existing standard/system, is one that:

• Allows to exchange data necessary for completing one or more procedure in the given domain
• (potentially) is already in use in multiple countries and adopted for cross-border data exchange
• Is based (as far as possible) on internationally recognised standards, hence ensuring a higher degree of interoperability

It is possible that in a given domain and for a given OOTS procedure, multiple standards could be identified and reused simultaneously, in order to retrieve all the information required by the OOTS (e.g. in the Education domain, EMREX and ELM).

Overall, this analysis can lead to one of the following main scenarios:

• Scenario 1: it has been identified a related system (such as EMREX or EUCARIS) which is already in use for cross-border data exchange in a specific domain. 
  Such related system assumes relevance in this activity since:
  • • i) it can be used as a point of reference for creating missing information requirements;
      or
    • ii) the existing information requirements should be analysed in order to ensure semantic compatibility with the related system.
      
      
• Scenario 2: no related system has been identified.
  In this case the generation of new Information Requirements and underlying concepts has no specific constraint connected to related systems, but is only bound to the needs/requirements expressed by Member States/Competent Authorities.

Such analysis, to be performed by the sub-group team (Editors and Coordinator), will be first shared and discussed with other relevant sub-group teams, and in particular those involved in the technical design of the OOTS (i.e. Specifications / TDD sub-group team). The purpose is to refine and validate this initial assessment by taking into consideration also the technical implementation side, since the beginning.

After reaching internal alignment among the relevant OOTS teams , the analysis will be  shared with the policy DGs and presented to the sub-group members, in order to collect any additional inputs and obtain validation of the findings (in case of Scenario 1, which system can be considered a 'related system'; in case of Scenario 2, agree on the absence of a 'related system').

Tool(s)
An online collaborative tool (e.g. Confluence) to share relevant documentation with the sub-group

Step 3 - Identify additional complementary standardisation efforts (optional)

Objective

Identify additional standardisation efforts which could complement the main standards and/or systems, in case a related system or a reference model is not capable of providing all the information needed to be exchanged via the OOTS. 

Key activities

• The sub-group team (Editors and Coordinator) analyses the information on existing systems/standards collected in Step 1, and identifies any potential related system or reference model
• The sub-group team (Editors and Coordinator) shares the outcome of the analysis information with the relevant policy DGs and with the sub-group members, for validation

Description

Through the preliminary analysis that allowed to identify a related system or an existing reference model to be reused in OOTS, the sub-group team will have already identified any high-level gaps the ability of such systems or models to in the related system to cover the OOTS requirements previously identified. 

To address such gaps, the sub-group team will try to identify other systems or models that might provide the missing information. This will be done in collaboration with the relevant policy DGs and subsequently the sub-group members will be informed and consulted in order to reach consensus.

For example: in the Education domain, while ELM is a reference model and EMREX is a relevant system currently allowing data exchange, both are currently presenting gaps when it comes to student mobility. For this purpose, additional systems like Erasmus Without Paper (EWP) could be taken into consideration to complement the information that ELM and EMREX are capable to provide.

Tool(s)
An online collaborative tool (e.g. Confluence) to share relevant documentation with the sub-group.

Phase 2: Mapping of OOTS requirements with reference models

This phase entails the existence of a related system (acting on behalf of the evidence provider) which can already cover the Information Requirements (meaning that such information requirement is already present and described in the related system data model). Therefore, in case of no related system, the process will move directly to Phase 3.

Accordingly, Phase 2 includes the following steps:

• Step 4 - Semantic mapping

• Step 5 - Technical mapping

• Step 6 - Validation of the mapping findings with policy DGs and standard owners 

Step 4 - Semantic mapping

Objective
Clarify the extent to which the information provided by a reference data model matches the information requirements of OOTS, previously indicated by the Member States as part of the Evidence Mapping. The purpose is to verify the semantic match with the OOTS information requirements.

Key activities

• The sub-group team (Editors and Coordinator) conducts an in-depth analysis of the properties of the related system, mapping them to the descriptions and meaning of the OOTS requirements
• The sub-group team (Editors and Coordinator), based on the outcome of the mapping, suggests an action to be taken in case of an imperfect match between OOTS information requirements and the properties of the reference data model, on a case-by-case basis   
• The sub-group team (Editors and Coordinator) shares the outcome of the mapping with relevant policy DGs and standard owners

Description

While a reference data model has the potential - in broader terms - to provide information related to a specific evidence, misalignments at semantic level (at the level of information requirement) can lead to errors in the data exchange and undermine the feasibility of the technical interconnection between OOTS and a related system.

Therefore it is necessary to conduct an in-depth semantic mapping to verify how each concept in a reference data model is expressed in comparison to the meaning given to each OOTS information requirement by the sub-group members (national coordinators and domain experts), in the context of the Evidence Mapping. The purpose is therefore to establish the relationships between the two.

Overall, the relationships between different attributes of a reference data model and the OOTS information requirements can be given one of the following values (based on the SKOS  - Simple Knowledge Organization System - mapping system), as described below (where RS stands for 'related system' or ' reference data model):  

Each one of the above described relationships is described in the following paragraphs.

4.1.1. Semantic 'perfect' match (point 1.1 of decision tree)

A semantic match is found out when the information requirement defined in OOTS matches, in terms of meaning, with the piece of information within the evidence provided by the related system. 

• For example: the "given name" of the owner of a motor vehicle is required by OOTS and it has the same meaning of the "given name" of the owner in the vehicle information provided by EUCARIS.

When a semantic match is found, then a technical analysis can already be conducted before the communication between OOTS and the related system takes place.

4.1.2. Semantic 'Broad' match (point 1.2 of decision tree)

A semantic broad match emerges when the information requirement from the related system is broader, in terms of meaning, than the information requirement in OOTS.

This means that the information provided via a reference model (and a related system) might provide a wider set of information, including information not required in the context of the OOTS.

• For example: the "LearningOpportunitySpecification" element in the ELMO model (normally related to the Diploma Supplement) could be a programme or a course, while in the OOTS Diploma Supplement requirement, among the properties, there is an explicit mention of the programme and the courses. That means there must be a way to distinguish them. In this specific case in the ELMO model, it is possible to distinguish thanks to the property "type" which could be a Degree Programme, Module, Course or a Class.

The problem exists when a related system does not provide a way to distinguish meanings, thus OOTS would not to be able to verify the information provided.

This case should be brought by the sub-group team to the owner of the related system, to explore a possible a way to distinguish the meanings.

4.1.3. Semantic 'Narrow' match (point 1.3 of decision tree)

A semantic narrow match is found out when the information requirement in OOTS, in terms of meaning,  is broader than the related system.

This means that the information provided via a reference model (and a related system) might provide only a very specific set of information which is a subset of the information required by the OOTS. 

• For example: in the Education domain, the current  OOTS information requirement "stream", associated to the "Proof of Secondary School course grades transcript completed in progress",  has a broader meaning than in the ELMO model (where very specific classifications are indicated, namely "subjectArea" and "iscedCode").

In this case, there are different options:

• the OOTS could simply accept what the related systems send over (after conducting a technical analysis to assess feasibility).
• a possible extension of the reference model (and scope of the related system) could be discussed with the owner

4.1.4. Semantic 'Partial' match / No match (point 1.4 of decision tree)

A semantic partial match or no match are found out when the meanings among OOTS and related system are different or partially common.

• For example: in the Education domain, the OOTS information requirements "status of the awarding education institution" or the "status of the education institution administering the studies" cannot be found in the ELMO model (even if it provides an "extension" element where a Member State could use it to add such information).

This case should be addressed according to the following actions:

• action 1: explore with the owner of the related system a possible extension of the model 
• action 2: verify the possible coverage of the information requirement by an additional/complementary model/system identified as part of Phase 1 (Step 3)
• action 3: bring the specific information requirement first to the policy DGs and then to the Member States (especially when extensions are not possible in the related system), to verify the rationale behind it and agree on how to address this case.

Tool(s) 

• This step is performed using a spreadsheet (Microsoft Excel), in which all the OOTS information requirements are listed in a column and is associated to the relevant attribute of the reference data model in a second column. The relationship between the two (narrow/broad/perfect/no match) is then specified in a third column. The mapping is then enriched by including the URI and the description of each information requirement, agreed upon as part of the evidence mapping.
  

Step 5 - Technical mapping

Objective
Identify the technical implications of mismatches and gaps identified through the semantic mapping , which can have an impact at technical implementation level and generate interconnectivity issues.

Key activities

• The sub-group team (Editors and Coordinator), as part of the  mapping, identifies and describes potential issues emerging at technical level  
• The sub-group team (Editors and Coordinator), presents the findings to the OOTS teams in charge of the technical implementation as well as to the owners of the related system 

Description

In this step, it is important to evaluate all possible issues and resolutions that can have an impact on the implementation of the communication between OOTS and related system.

This activity is not part of the data semantic standardisation process per se, but it is crucial in order to ensure a smooth exchange of structured evidences through the OOTS.

When such issues are detected, these are communicated to the OOTS teams in charge of the technical implementation (normally the Specifications/TDD sub-group) as well as to the owners of the related systems, for them to decide the best way to address such issues.

There are several types of issues that might emerge. The following table provides a non-exhaustive list of examples of possible ones:

Tool(s)

• A spreadsheet (Microsoft Excel), to perform the mapping and annotate findings (same as Step 4)
• Collaboration tools such as Microsoft Teams could be used to share and review working drafts of the mappings with other OOTS sub-group teams and with owners of related systems. 

Step 6 - Validation of the mapping findings with policy DGs and standard owners

Objective
Communicate the results of the mapping to the Member States, including proposed solutions to gaps and issues identified, to collect feedback and validate the mappings.

Key activities

• The sub-group team (Editors and Coordinator) prepare and share the relevant documentation and present the outcome of the mapping to the policy DGs and standard owners
• The policy DGs and standard owners review the mapping and provide feedback and clarifications.
• The sub-group team (Editors and Coordinator) refines the mappings and shares them again with policy DGs and standard owners for validation

Description

Once the mapping have been completed, these can be shared with the policy DGs and the standard owners for review (in some cases, these two can coincide, such as in the case of ELM in the education domain).

The policy DGs and standard owners, acting as subject matter experts (in the domain and on the reference data model), provide clarifications or suggestions for further refining the mapping.

The sub-group team therefore finalise the mapping based on the inputs received and shares it with policy DGs and standard owners for final validation.

At this point, the validated and finalised mappings can be shared with the sub-group members for their review and comments.

Tool(s)

• A spreadsheet (Microsoft Excel), to perform the mapping and annotate findings (same as Step 4)
• Collaboration tools such as Microsoft Teams could be used to share and review working drafts of the mappings with other OOTS sub-group teams and with owners of related systems
• An online collaborative tool (e.g. Confluence) to share relevant documentation with the sub-group

Phase 3: Complete the semantic expression of OOTS requirements

This phase - which is relevant both when a related system exists and when it has not been identified - includes the following steps:

• Step 7 - Identification of ontologies 

• Step 8 - Mapping an Information Requirement to a class/property

Step 7 - Identification of ontologies 

Objective
List all relevant ontologies to be potentially reused for collecting concepts relevant for describing the OOTS information requirements.

Key activities

• The sub-group team (Editors and Coordinator) prepares and shares a list of reference ontologies to be used for a specific domain, in addition to pre-defined transversal 'cross-domain' ones
• The policy DGs and standard owners review the list and provide feedback and suggestions
• The sub-group members (supported by the domain experts) review the list and provide feedback and suggestions

Description

All the Information Requirements - once MS have agreed on their specific 'meaning' - have to be assigned a vocabulary that will be used to semantically preserve such meaning and ensure accuracy in the data exchange,

In order to do this, it is necessary to identify the potential ontologies from which it will be possible to pick the correct vocabulary. Afterwards, MS will be involved in order to validate the proposed vocabularies or identify alternative ones.

It is possible to distinguish between two macro-categories of ontologies:

• Main 'transversal' reference ontologies
  Reference ontologies are existing data models that can be used horizontally across multiple domains. By looking at existing ontologies reused at international and European level it is possible to determine the following list:

• • SKOS, a W3C Recommendation, used mainly when properties have a determined number of values/controlled lists (such list of countries or languages)
  • FOAF, a wide used community specification, used mainly to describe concept as Person (that can be real or not, from which Core Person is derived), Organization or a Document.
  • DC Elements and DC Terms,  DCMI recommendations, to define common properties such as title, description, etc.
  • Organization Ontology, a W3C Recommendation, providing the basis to describe an Organization and related to FOAF, from which Core Business and Core Public Organization are derived.

This list will be updated if additional ontologies are needed in common among the domains.

There is no order/priority among them, they need to be evaluate over time.

• Domain-specific ontologies
  For each domain, additional reference vocabularies might be needed to be evaluated.

In cases when a related system has been identified, specific ontologies developed in the context of those systems could also be considered for assigning a vocabulary to OOTS requirements. However, this possibility is considered only when no other ontology provides the required vocabulary; therefore trying as much as possible to keep the chosen set of vocabularies 'system-agnostic'.

When multiple domain-specific ontologies have been identified, the following criteria are to be adopted in order to make the optimal choice:

• • Scope: Priority to international ontologies
  • Relevance: domain-specific and complementary to the transversal ontologies
  • Compatibility: No potential conflict or with generic/horizontal ontologies
  • Maintenance: prioritising ontologies that are regularly updated

Example: in the education domain, ELM is the ontology that can be considered as relevant as it is European, fills the most of the OOTS requirements, it is compatible with FOAF (Agent, Group), SKOS for controlled list, and it is maintained by DG EMPL.

Tool(s)

• A spreadsheet (Microsoft Excel), to map all the relevant ontologies and track dependencies between them
• An online collaborative tool (e.g. Confluence) to share relevant documentation with the sub-group and collect feedback.

Step 8 - Mapping an Information Requirement to a class/property

Objective
Ensure that all OOTS information requirements have been assigned a specific concept that preserves their agreed meaning.

Key activities

• The sub-group team (Editors and Coordinator) identify and propose a concept to be assigned to an OOTS Information Requirement
• The sub-group members (supported by national domain experts) provide feedback and/or validate the proposed concept

Description

After completing Step 7, it is therefore possible to proceed to assigning a concept to an OOTS Information Requirement.

This activity is driven by the following research question: 'does the OOTS information requirement have a correspondent concept (class/property) in an existing ontology, that has the same meaning?'

In answering such question, priority will be given to concepts and vocabularies already included in the main 'transversal' reference ontologies. Moving to domain-specific ontologies in case of negative result of the search in reference ontologies. Should also the search into domain-specific ontologies provide a negative result, it will be necessary to propose the creation of a new concept, specific for OOTS.

The diagram below describes the steps that will be followed in order to identify a relevant vocabulary that can be proposed for reuse under OOTS, to describe information requirements.

For each Requirement, the Editor will prepare a spreadsheet listing all the information requirements and the chosen vocabulary associated to them.

Tool(s)

• An online collaborative tool (e.g. Confluence) to share relevant documentation with the sub-group and collect feedback.
• A spreadsheet (Excel) for listing all the chosen vocabularies

Phase 4: Draft and validate the data model

This phase includes the following steps:

• Step 9 - Combine the OOTS requirements and evidence types into a data models based on CCCEV 
• Step 10 - Publish draft data model
• Step 11 - Review and validate draft data model with sub-group

Step 9 - Combine the OOTS requirements, evidence types and mappings into a data model based on CCCEV

Objective
Structure the OOTS requirements including the mapping in the overall OOTS data model, according to the Core Criterion and Core Evidence Vocabulary (CCCEV).

Key activities

• The sub-group team (Editors and Coordinator) consolidates the requirements, evidence types and mappings in a data model based on CCCEV

Description

Phases 1 to 3 allowed to: i) identify a related system and reference data model; ii) map the way in which it covers the information requirements related to an evidence type; and iii) describe each information requirement using vocabularies from reference ontologies.

Accordingly, the previous Phases allowed to map information requirements (which combined make up OOTS requirements), by analysing and mapping each requirement separately. However, given that in order to provide an evidence, there might be the need to combine multiple requirements, these have to be put together in the same model which brings together structured data related to multiple requirements.

In order to bring together all these elements into a coherent framework and start drafting a data model, the Editors will base their preparatory work on existing European standards and, in particular, reuse the Core Criterion and Core Evidence Vocabulary (CCCEV) provided and regularly maintained by SEMIC and aligned to recognised international standards.

CCCEV is a European reference vocabulary developed with the purpose to describe information requirements and evidences. In particular, it includes core concepts such as 'requirement' and 'evidence'.

Therefore it has been identified as the reference schema for modelling data related to OOTS requirements and evidence types.

Accordingly, the sub-group team (Editors and Coordinator) reuses the CCCEV to describe:

1. the OOTS requirements as instances of Requirement class at different granular level (e.g. the  "Request for Proof of tertiary education diploma/certificate/degree" aggregates "given name","family name","name of qualification", etc.) with their related concepts from the ontologies identified in Phase 3 (Step 8).
2. the instances of Information concept as result of the mappings in the previous phase (e.g. "given name" with its expression of expected value)
3. the instances of Evidence Type List to group together multiple Evidence Types (e.g. "List for Proof of tertiary education diploma/certificate/degree")
4. the instances of Evidence Types (e.g. "tertiary education diploma/certificate/degree")
5. In addition, the sub-group relates, for each information concept, the mapping found from the previous phases (Phase 2 and 3).

All the above-listed descriptions are included in a spreadsheet that brings together the OOTS Requirements with the reference data model (of the related system) and the results of the mapping conducted in the previous Phase.

Therefore the spreadsheet prepared in this Step represents a mean for justifying the chosen structure and attributes of the data model that the sub-group team will draft for each domain (education, vehicle, etc.)

This Step corresponds to preparatory work conducted internally by the sub-group team, based on outputs previously validated by Member States. The output of this Step will be subsequently presented and discussed with Member States, as described in the following Steps.

Tool(s)

A spreadsheet structured according to CCCEV is created.

Step 10 - Test and publish draft data model

Objective
Make the draft data model publicly available to the Member States.

Key activities

• The sub-group team (Editors and Coordinator) converts the spreadsheet from the previous step to the respective RDF   
• The sub-group team (Editors and Coordinator) test the RDF against the CCCEV for conformance
• The sub-group team (Editors and Coordinator) publishes the spreadsheet and the respective RDF into the Semantic Repository

Description

The spreadsheet created in the previous step is transformed into RDF according to the CCCEV data model. The Editors proceed to verify the conformance of such RDF against CCCEV.

• If it is conformant, the sub-group team publishes the spreadsheet and the respective RDF into the Semantic Repository.
• If it is not conformant, the sub-group team adjust the spreadsheet until the conformance is reached. 

Tool(s)

• Skos-Play! service is used to transform the spreadsheet into RDF.
• pySHACL or Jena SHACL tools can be used for conformance testing against CCCEV

Step 11 - Review and validate draft data model with the sub-group

Objective
Conduct iterations with the Member States, in order to validate the proposed data model.

Key activities

• The sub-group team (Editors and Coordinator) prepares and shares the relevant documentation with the sub-group members
• The sub-group members and the domain experts review the draft data model and provide feedback and clarifications.
• The sub-group team (Editors and Coordinator) refines the data model, in collaboration with the standard owners - shares it again with sub-group members for validation

Description

The sub-group team (Editors and Coordinator) prepares the documentation on the model created and present it to the sub-group members and domain experts for review.

Once feedback is received, the model is reviewed and updated by the sub-group team.

Tool(s)

• Collaboration tools such as Microsoft Teams could be used to share and review working drafts of the data model with other OOTS sub-group teams and with owners of related systems
• An online collaborative tool (e.g. Confluence) to share relevant documentation with the sub-group

Phase 5: Select controlled lists

This phase includes the following steps:

• Step 12 - Identify and propose controlled lists across the model

• Step 13 - Choose recommended controlled lists 

• Step 14 - Create new controlled lists (optional)

• Step 15 - Document core sets of attributes and recommended controlled lists

Step 12 - Identify and propose controlled lists across the model

Objective
Analyse controlled lists for common attributes in the data model drafted in Phase 4.

Key activities

• The sub-group members and the domain experts propose controlled lists for the different attributes defined in the previous phases.
• The sub-group team (Editors and Coordinator) synthesise the propositions and complement with additional standard controlled lists where relevant.

Description

Once a core set of common attributes has been agreed upon and the draft OOTS data model for specific evidence types is stable enough, the set of controlled lists, for those attributes where a controlled lists is needed, has to be analysed.

The source of these controlled lists can be different:

• those coming from the OOTS requirements (for example the domain categorization (education, vehicle, etc.) or the list of countries associated to an Evidence Type)
• those proposed by the related systems (for example, in the Education domain, the ISCED classification is used to identify a Learning Opportunity in the ELMO XML schema) 
• those proposed by existing ontologies (for example, in the Education domain, the ELM model has already published controlled lists at the Publications Office such as Accreditation decisions, Assessment type, etc.)

Along with the controlled lists, the sub-group is tasked with proposing usage notes for all the attributes agreed upon.

Rules and Guidelines

• Controlled lists should have governance processes in place, be hosted in a sustainable manner and be provided free of charge.
• Controlled lists at the EU level are multilingual which helps in cross- border data exchange scenarios.
• (Domain-specific) Controlled lists which are internationally accepted should be considered.

Tool(s)

• An online collaborative tool (e.g. Confluence) to share relevant documentation with the sub-group and collect feedback.

Step 13 - Choose recommended controlled lists

Objective
Assess and recommend suitable controlled lists.

Key activities

• The sub-group team (Editors and Coordinator) puts forward the different propositions for each attribute working towards a decision.
• The sub-group members and the domain experts discuss - through the collaborative tool - and select the controlled lists.
• The sub-group members and the domain experts review the proposed controlled lists and provide feedback and suggestions.
• The sub-group team (Editors and Coordinator) assesses the feedback and suggestions received (forwarding them to the EU Publication Office when needed) and, once ready, shares again the controlled lists with sub-group members for validation

Description

Based on the table of controlled lists, the sub-group members discuss which controlled lists are the most appropriate to be recommended. They also review whether the proposed usage notes are adequate. This may be based on the status of particular lists (e.g. if they are based on an international standard) or on their usage across multiple implementations.

It is important to agree on common official controlled lists that can harmonise the way in which specific values of properties are specified across different countries, allowing for a uniform indexing and retrieving of data based on common terms.

Once the sub-group team has consolidated the controlled lists to be proposed for reuse in the data model (or when a draft list prepared by the EU Publication Office has been drafted), these are shared with the sub-group members for their review and comments.

In this phase, iterations within the sub-group take place until there is consensus on the controlled lists to be used and the final selection is validated by the sub-group members.  In the case of divergent views, a live discussion (sub-group meeting) may be organised by the Editors and the Coordinator to arrive at a consensus on the most controversial proposed solutions.

Tool(s)

• An online collaborative tool (e.g. Confluence) to share relevant documentation with the sub-group.

Step 14 - Create new controlled lists (optional)

Objective
Propose the creation of new controlled lists in the absence of existing ones.

Key activities

• The sub-group team (Editors and Coordinator) create a proposition of a new controlled list.
• The sub-group members review the proposition and provide comments.
• The EU Publication Office creates a controlled list.
• The sub-group members and the domain experts review the proposed controlled list and provide feedback and suggestions.
• The sub-group team (Editors and Coordinator) assesses the feedback and suggestions received (forwarding them to the EU Publication Office when needed) and once ready shares again the controlled list with sub-group members for validation

Description

In the event of no controlled list being available, the sub-group team (Editors and Coordinator) or sub-group members) have the opportunity to propose the creation of new controlled list. Required controlled lists that do not yet exist, will be created the sub-group team and then published by by the Publications Office, as part of the EU Vocabularies. If necessary, existing controlled lists can be updated.

Once the sub-group team has consolidated the controlled lists to be proposed for reuse in the data model (or when a draft list prepared by the EU Publication Office has been drafted), these are shared with the sub-group members for their review and comments.

In this phase, iterations within the sub-group take place until there is consensus on the controlled lists to be used and the final selection is validated by the sub-group members. In the case of divergent views, a live discussion (sub-group meeting) may be organised by the Editors and the Coordinator to arrive at a consensus on the most controversial proposed solutions.

Tool(s)

• An online collaborative tool (e.g. Confluence) to share relevant documentation with the sub-group and collect feedback.
• A spreadsheet (e.g. Excel) to be shared with the EU Publication Office to list all the controlled lists that need to be created or its RDF expression.

Step 15 - Document core sets of attributes and recommended controlled lists

Objective
Outline the essential requirements an implementation must meet to align with a particular OOTS data model.

Key activities

• The sub-group team (Editors and Coordinator) document the consensus and construct the working draft.

Description

On the basis of the previous steps of this Phase and according to the outcomes of Phases 3 and 4, the Editors will document the decisions and prepare to update the draft OOTS data model for specific evidence types.

As part of this Step, the Editors will:

• Link of the controlled lists with the properties of the data mode; this will be done by creating a table which associates each controlled list to the relevant concept of the model (an example can be found in the 
• Describe the core set of attributes for the data model in both machine-readable and human-readable format

Once this activity is completed, all the controlled lists and the relevant documentation will be uploaded in the OOTS Semantic Repository, to complement the assets already uploaded as a result of Phase 4.

Phase 6: Create distributions and publish documentation

This phase includes the following steps:

• Step 16 - Decide on the conformance requirements and draft a conformance statement

• Step 17 - Create distributions

• Step 18 - Publish all documentation

• Step 19 - Test the final data model with instance data
• Closing

Step 16 - Decide on the conformance requirements and draft a conformance statement

Objective
Outline the essential requirements an implementation must meet to align with a particular OOTS data model.

Key activities

• The sub-group members (Editors and Coordinator) draft a conformance statement.
• The sub-group members agree on the conformance statement.

Description

A conformance statement declares a minimum set of requirements that an implementation must adhere to, in order to be considered conformant with the respective OOTS data model for specific evidence types. The sub-group members must agree on these conformance requirements. The Editors then include a conformance statement in the OOTS data model for specific evidence types.

The OOTS data model for specific evidence types may have natural divisions, in which case it might be appropriate to set different conformance levels. For example, a model used to describe vehicles may have a group of terms related specifically to motor vehicles that could be used in an implementation that has no need to understand the terms that relate to bicycles. This will consequently lead to the establishment of different conformance levels.

The conformance statement has to be published together with the OOTS data model for specific evidence types.

Tool(s)

• An online collaborative tool (e.g. Confluence) to share relevant documentation with the sub-group.

Step 17 - Create distributions

Objective
Create distributions for the data model in different formats.

Key activities

• The sub-group Editors create the required distributions for the data model for specific evidence types.

Description

The OOTS data model for specific evidence types can be expressed (or serialised) in various formats depending on the specific needs and context. Each distribution (format) will have its own uses and advantages, but also its own disadvantages and limitations.

Semantic data models can be expressed in different serialisation formats. Special care needs to be taken when using multiple formats, as conversion between different serialisation formats can potentially introduce inconsistencies. During this step, URIs are also created (or reused when possible) for the data model for specific evidence types itself, its entities and their attributes. These identifiers need to be minted and maintained by a (European Commission) service.

In addition, the OOTS Semantic Repository will provide instructions on the modalities for making available distributions.

Aside from machine-readable formats, human-readable formats also need to be created and human-readable documentation is also required with all the necessary information to construct the data model (i.e. the entities and attributes with their definitions, cardinalities, proposed codelists, etc.).  This can be distributed as an HTML-page and a PDF-document, for example.

Rules and Guidelines

• Create both machine-readable as well as human-readable distributions of the data model.
• Use URIs under data.europa.eu which allows as to flexibility for where the URIs resolve to.
  
  

Tool(s)

The data model and its documentation will be published in the OOTS Semantic Repository.

Step 18 - Publish all documentation

Objective
Make ready all the documentation related to the data model drafted in the previous phases in the OOTS semantic repository

Key activities

• The sub-group Editors publish all documentation on the collaborative tool.

Description

The Editors publish the final version of the OOTS data model for specific evidence types, in both machine-readable and human-readable formats in the OOT Semantic Repository. The Editors must publish the OOTS data model for specific evidence types as open (meta)data and specify which license is applicable.

Tool(s) 

• An online collaborative tool (e.g. Confluence) to share relevant documentation with the sub-group
• OOTS Semantic Repository

Step 19 - Test the final data model with instance data

Objective
Verify the technical feasibility of structured data exchange based on the model that has been profiled and validated in the previous Phases.

Key activities

• A selected number of sub-group members and domain experts test the model against instance data, under the supervision of the OOTS technical sub-group teams.
• The Editors assist the sub-group members in the testing activities by collecting and categorising the feedback.

Description

So far, the process of defining the elements of the OOTS data model for specific evidence types was a theoretical exercise. The objective of this step is to test the final model against instance data, i.e. actual data and will be conducted in collaboration with other sub-group teams in charge of the technical implementation (mostly the Specifications / TDD sub-group team), which will lead the execution of such tests with the support of the data models standardisation sub-group.

Testing the data model will allow to discover potential flaws or blind spots in the model. In this step, sub-group members have to provide (dummy) instance data and report on the challenges they face when:

• mapping this instance data to the model (perspective of the data provider). Sub-group members must answer the question: “Can we provide this information?”.
• processing instance data that respects the OOTS data model for specific evidence types (perspective of the data consumer). Sub-group members must now answer the question: “Can we process this information?”, where the information represents the minimum data required by the model and, in this case, considering that the data was hypothetically received from another party.

A likely process for this step could be as follows:

1. Initiate – All sub-group members have the possibility to volunteer for the testing of the OOTS data model for specific evidence types with instance data. At the beginning of this exercise, Editors will organise a meeting with the volunteers to walk them through the process and outline the expectations.
2. Map – Some volunteers will play the role of the Evidence Provider and create instance data for the OOTS data model for specific evidence types, with as many attributes as are available in their national system, and map them to the attributes to the data model to be tested.
3. Execute – Some volunteers will play the role of the Evidence Requester and receive minimal evidence (mandatory fields only) data from a sub-group member playing the role of Evidence Provider. These volunteers will then process the instance data received.
4. Report – The sub-group members participating in the test will report on (semantic) challenges and errors arising from test exchange of instance data. This step should reveal potential flaws in the model thanks to the life-like situation of processing an evidence.
5. Improve – The Editors will take stock of the findings of this test in order to improve the model (e.g. by adding usage notes, reviewing the mapping and updating documentation in the OOTS Semantic Repository and in the Evidence Broker). After that, the Editors share the updated model with the sub-group members for their review and validation.

The test might be conducted according to different cases (listed in the 'Introduction' paragraph) and depending on the whether Evidence Requester and Evidence Provider are connected or not to a related system.

Tools

• Mapping spreadsheet (Excel) prepared in Phases 2 and 3.
• OOTS Semantic Repository and Evidence Broker

Closing

Step 19 corresponds to the last step of the OOTS data models standardisation methodology.  

The Phases and steps described in this methodology correspond to the standard process for profiling data models for completing public procedures in the scope of the OOTS.

Variations to Phases and Steps described can take place, depending on the specific domain of standardisation and stakeholders involved.

In case of subsequent updates of the data models drafted according to this methodology, specific Phases and Steps might be conducted again, possibly without replicating the entire process.

In any case, all the data models, the related semantic assets (controlled lists) to be included in the Semantic Repository will have to be previously validated by the sub-group members. This is valid also in case of subsequent updates.