Page tree
Skip to end of metadata
Go to start of metadata

Table of contents

1. Purpose

The purpose of the end-to-end functional testing is to validate, from the User point of view, the process and the information provided to HP (Health Professionals).

The evaluation is performed for the three services (Patient Summary, ePrescription and eDispensation) in an environment that intends to emulate the normal operation as much as possible (e.g. a pharmacist dispensing a medicinal product to a patient from a different deploying country); the only difference being that only test patients and test data are used and no real patients are involved.

Health professionals (physicians and pharmacists) together with semantic experts perform the test and fill in an online questionnaire (the CDA Evaluation Form) that allows them to evaluate the process and whether the CDA documents exchanged and translated by the NCP-B contains the expected information.


2. Methodology

Overall procedure

The functional testing is based on the analysis of the received document by a HP in Country B, when he requests a PS/eP for a specific patient, or the returned eD retrieved in Country A (following the dispensation of an eP in Country B).

The test tool is the online CDA Evaluation Form, which contains various evaluation sections: non-functional evaluation, administrative evaluation, structural evaluation, and medical/pharmaceutical evaluation.

  • Non-functional evaluation: evaluation of the response time between NCPs (informative);
  • Administrative evaluation: evaluation of the information related to the patient, the context of the CDA document;
  • Structural evaluation: related to the relationship of the CDA document to the other CDAs (like the PDF version, etc);
  • Medical/pharmaceutical evaluation: an evaluation of the medical/pharmaceutical content of the CDA document. This provides an overview of the safety of the process, allowing evaluating if the medical content can be understood by a physician after its translation or the electronic prescription by a pharmacist.

A brief summary of the overall procedure is shown in the diagram below:

Credentials to access the CDA Evaluation Form

Access to fill in and submit the online questionnaire - the CDA Evaluation Form - for the Functional testing requires an account in gazelle.ihe.net (not the same as for the testing platform).

If you did not attend the Second Semantic Boot Camp in Brussels (May 16-17) you may need to request an account from IHE (abderrazek.boufahja@ihe-europe.net)

Background information for the Health Professionals participating in the e2e functional testing

Preparatory steps

  • Prepare and make available Test Data (as L1 and L3 CDA documents):
    • Critical Test Data, i.e. test data agreed by experts, describing an agreed clinical history, to verify interoperability capability.
    • Representative Test Data, i.e. documents representing typical PS or eP documents provided by your national infrastructure.
  • Provide patient identifiers to be used by other countries to retrieve clinical information (the Test Data Files mentioned above). Please, distinguish between patient identifiers for Patient Summary and for ePrescription documents. Include this information in the table available in the Test Data page (that also shows the services provided by each deploying country).
  • In the case of eDispensation, identify countries that will use your ePrescription service (check the same table above or the one on this page) to give feedback on the eDispensation document received.
    • To access an eDispensation document, ask your technical team how eDispensation documents are processed when received in your country.
  • Include name and contact details about the HP and Semantic experts participating in the evaluation in the table of eHDSI Deploying Countries (bottom of this page).
    • Providing these contact details will allow asking the counterparts any doubt about the documents received.
    • Additionally, you may use Slack as a way to establish immediate communication with other participants and specially with your counterpart in the evaluation (Country-A in the case of PS and eP documents; Country-B for eD documents): in https://openncp.slack.com/ a channel is dedicated to the functional testing (requires creating an account and be invited or request to join).
  • Obtain the necessary credentials to log in to your National Health Portal (the information system that enables the access to patient from other deploying country's information) with the appropriate role (physician/pharmacist).

Test Session Procedure and Post-testing Activities

STEPDESCRIPTIONPERFORMED BYTOOLSREMARKS
Introduction and Context creationProvision of background information (e.g. what are the eHDSI services, eP and PS workflow, considerations on cross border care, the structure of the documents and its content, MVC and MTC Value Sets, how known absent and missing information is represented) enabling the health professional with context and broader awareness on what he will face when using the cross border eHealth information services.Country Semantic Expert

Presentations

Special focus on workflows, check points and decisions that the health professional needs to take and can be accounted for.
Log in to Health PortalThe HP participating in the test logs in using his credentials for the Health PortalCountry Health ProfessionalPortalDepeding on the role (physician/pharmacist), the HP will access PS or eP documents
Perform Test

Using the tools and data available, the physician/pharmacist retrieves the clinical data for the "virtual" patients that are presented to him. For the ePrescription service, the pharmacist simulates a dispensation and introduces the necessary information about the dispensed medicinal product.

(info) A deploying country testing PS services as Country-B has to test with all deploying countries acting as Country A. Likewise, a deploying country acting as Country-A has to be tested by all Country-B for the PS service.
Country Health ProfessionalPortal + NCP + Critical Test Data + Representative Test DataClear instructions on who are the patients and how they can be identified should be available so that the health professional can perform the workflows without additional burden.
Feedback Dialogue

Gathering Health Professional findings

(lightbulb) You may consider useful contacting the counterpart in the evaluation for any clarification; either using email or Slack for immediate communication (https://openncp.slack.com/).
Country Health ProfessionalCountry specificFeedback should be documented (Up to the country to decide how to collect and document it: e.g. screenshots inserted in a word document could greatly help the subsequent post-testing analysis of the findings)
Questions

Questions and Answer Session from Health Professional to the Semantic Expert

(info) Specially relevant is the feedback from the Health Professional about the representation of both missing information and known absence of information; did he/she correctly understood why the information was not present? (e.g. a situation when no information about allergies is available vs the fact that the patient does not have any known allergies)
Country Health ProfessionalCountry specificFeedback should be documented (Up to the country to decide how to collect and document it)
Feedback Submission

CDA Evaluation Form fulfilment and submission (optionally: the Health Professional does not need to be present for this step as it can be done by the Semantic Expert)


Country Semantic ExpertCDA Evaluation Tool

The person submitting the evaluation will receive an email with the content of the submission and a link to its html representation. Some deploying countries have proposed to share the evaluation with the counterpart in the exercise.

(info) At this moment the automatic notification (with the content) after each submission is only received by SP, but not implemented yet for the user that submits the evaluation.For this reason, SP will forward the notifications received.

Still, it is possible for registered users in Gazelle to consult all the evaluations submitted.

Evaluation ReportA consolidated view on the findings and feedback collected regarding tests in which your country  participatedeHDSI Semantic Test ResponsibleExcel file with findings
Post-testing ActivitiesAnalysis of the findings by the eHMSEG Semantic Task ForceeHMSEG Semantic Task Force (ad hoc operational work group)JIRA tickets if necessary; repository of findings on ConfluenceThe eHMSEG Semantic Task Force members proposed to organise a teleconference meeting once the functional testing is finalised to discuss the findings, create JIRA tickets if necessary as well as set up a repository of findings on Confluence.


3. eHDSI Deploying Countries

COUNTRYCOUNTRY CODESERVICES

Health Professional

Semantic Expert 
PS-APS-BeP-AeP-B

Luxembourg

LU
X

Philippe Rémy

Olivier Lepanto

Jean-Claude Karasi

Olivier Lepanto

Croatia

HR


XXX

Sweden

SE

X


Estonia

EE


X

Finland

FI

X


Portugal

PT

X

XXX

Cyprus

CY

X

X

Dr Vasos Scoutellas

Health Professional, Ministry of Health (VScoutellas@mphs.moh.gov.cy)

Dr Mary Avraamidou

Health Professional, Ministry of Health (MAvraamidou@mphs.moh.gov.cy)

Dr Vasos Scoutellas

Health Professional, Ministry of Health (VScoutellas@mphs.moh.gov.cy)

Dr Mary Avraamidou

Health Professional, Ministry of Health (MAvraamidou@mphs.moh.gov.cy)

Czech Republic

CZ

X

X

Petr Kajzar

Health Professional (account @Petr Kajzar, petr.kajzar@lf1.cuni.cz)

Hynek Kružík
Semantic Expert (account @Hynek KRUZIK, kruzik@gnomon.cz)

Malta

MT

X

X

Dr Aaron SCHEMBRI

Emergency Department, Mater Dei Hospital

aaron.schembri@gov.mt

Dr Hugo AGIUS MUSCAT

Information Management Unit, Ministry for Health

hugo.agius-muscat@gov.mt

Switzerland

CH

X







4. Outcomes

  • CDA Evaluation Report, per country (information is restricted to each country).
    • Meaning that the feedback collected about a Country A, is only provided to that Country A.
  • Analysis of the findings by the eHMSEG Semantic Task Force
  • No labels