Digital

Page tree

Skip to main content


Specifications

In this section:

eDelivery Conformance Testing Service 2023.06-p1


This page collects the resources for eDelivery Conformance Testing Service version 2023.06-p1, released in December 2024.

Description

We are happy to announce the release of eDelivery Conformance Testing Service version 2023.06-p1. It includes a number of new features, improvements and bug fixes:

  • Specifications:

    • Support for X509PKIPathv1 Token Type. It is one of the three mechanisms to reference a security token listed in section 3.2.6. Security of the eDelivery AS4 Profile v1.15.

    • Support for testing the ENTSOG AS4 Usage Profile Rev 3.5.

  • Testing Environment:

    • The platform checks that User Messages that contain incorrect actions according to the test case are rejected by the SUT.

    • Changes in the 'Deliver' action now allow the Platform to notify the test driver that SUT has received and processed the User Message.

    • The UI now correctly labels the Error Signal Messages test steps.

    • The UI now correctly distinguishes between the control commands "Command Deliver" and "Command Notify".

    • Upgraded the Interoperability Test Bed platform from version 1.21.0 to 1.24.4.

Test Cases

Solutions that were conformant according to the previous version of the conformance testing platform (2023.06) can easily become conformant according to this newer version. To do so, they must:

  • pass all new tests that are not marked as "optional"
  • pass all updated tests that are marked as "to be repeated"

New tests marked as "optional" and updated tests marked as "no need to repeat" do not have to be passed (again).

(AS4 1.15) Basic Connectivity Tests v1.1

Updated Tests - No need to repeat

    • (AS4 1.15) Basic One Way Retry C2

(AS4 1.15) AS4 Common Profile Tests v1.1

New Tests

    • (AS4 1.15) CP_TA47_C3 (Optional)
    • (AS4 1.15) CP_TA48_C2 (Optional)
    • (AS4 1.15) CP_TA49_C3 (Optional)
    • (AS4 1.15) CP_TA50_C3 (Optional)
    • (AS4 1.15) CP_TA51_C3 (Optional)

Updated Tests - To be repeated

    • (AS4 1.15) CP_TA08_C3

    • (AS4 1.15) CP_TA26_C3

    • (AS4 1.15) CP_TA29_C3

    • (AS4 1.15) CP_TA30_C3

    • (AS4 1.15) CP_TA31_C3

    • (AS4 1.15) CP_TA37_C3

Updated Tests - No need to repeat

    • (AS4 1.15) CP_TA09_C2
    • (AS4 1.15) CP_TA10_C2
    • (AS4 1.15) CP_TA15_C2

(AS4 1.15) ENTSOG Tests - v1.0

New Tests

    • (AS4 1.15) ENTSOG_TA01_C2

    • (AS4 1.15) ENTSOG_TA01_WITH_MPC_C3

    • (AS4 1.15) ENTSOG_TA01_WITHOUT_MPC_C3

    • (AS4 1.15) ENTSOG_TA02_C2

    • (AS4 1.15) ENTSOG_TA02_C3

    • (AS4 1.15) ENTSOG_TA03_C2

    • (AS4 1.15) ENTSOG_TA03_C3

    • (AS4 1.15) ENTSOG_TA04_C2

    • (AS4 1.15) ENTSOG_TA04_C3

    • (AS4 1.15) ENTSOG_TA05_C2

    • (AS4 1.15) ENTSOG_TA05_C3

    • (AS4 1.15) ENTSOG_TA06_C2

    • (AS4 1.15) ENTSOG_TA06_C3

    • (AS4 1.15) ENTSOG_TA07_C2

    • (AS4 1.15) ENTSOG_TA07_C3

    • (AS4 1.15) ENTSOG_TA08_C2

    • (AS4 1.15) ENTSOG_TA08_C3

    • (AS4 1.15) ENTSOG_TA09_C2

    • (AS4 1.15) ENTSOG_TA09_C3

    • (AS4 1.15) ENTSOG_TA10_C2

    • (AS4 1.15) ENTSOG_TA10_C3

    • (AS4 1.15) ENTSOG_TA11_C2

    • (AS4 1.15) ENTSOG_TA11_C3

    • (AS4 1.15) ENTSOG_TA12_C3

Impact on SUT Controller

A SUT Controller that was used with the previous version of the Conformance Testing Service (2023.06) must be updated in order to be used with this new version.

Interoperability Test Bed Platform version v1.24.4

We are happy to announce that we upgraded the Service platform from v1.21.0 to v1.24.4. Please find below the summary of the most important changes:

  • Redesigned conformance dashboard
  • Allow organisation users to consult conformance snapshots
  • Reports at the level of specification groups
  • Conformance badges in PDF reports

Detailed release notes for intervening ITB releases:


Technical Documentation


hammer and wrench  Guides
Conformance Testing Service User Guide


User Guide for using the Interoperability Test Bed as customized for the eDelivery Conformance Testing Service. Key content includes an explanation of the components of the test bed and the actions required from the software/service providers making use of the service.

Release notes

Please find below the list of new features, improvements, and Fixed bugs.

New Features:

  • TestDriver now validates the Action in the UserMessage. Action Deliver is now used to notify the Test Driver that SUT has received (and processed) a user message.
  • Added support for X509PKIPathv1 Token Type. It is one of the three mechanisms to reference a security token listed in section 3.2.6. Security of the eDelivery AS4 Profile v1.15.
  • Added support for testing the ENTSOG AS4 Usage Profile Rev 3.5.

Improvements:

  • AbstractBackendConnector#deliverMessage awaits for the signalMessage to reach TestDriver and be processed, before processing the Deliver/Notify messages for test cases where SUT is C3.
  • TestDriver now validates the Action in the UserMessage. Action Deliver is now used to notify the Test Driver that SUT has received (and processed) a user message.
  • SUT Controller's remaining references to Kerkovi were renamed.
  • Test steps timeout implemented.
  • UserMessage validations were added to situations where there is a notify to the producer when SUT is C2.

Fixed Bugs:

  • Safeguard against NullPointerException thrown when trying to access Test Case properties, when the Test Case had executed and the session already finalized in GitbBackendConnectorImpl.
  • Removed noReliability usage in the PMode which led to wrong behaviour regarding Receipts.
  • UserMessage MessageProperties coming from SUT had repeated Property elements containing both lower and upper case in the initial letter of the Property name.
  • Payload was missing cid: as prefix.
  • Test steps report showed incorrect time (server startup time). Test Steps Report's Envelope label "INPUT" was changed to "Message Envelope".
  • Test step reports fixes: timestamp corrected, label for the XML Message corrected, test behaviours section not displayed anymore if there are no associated test behaviours.
  • Signal Error Messages for Retry Test Cases should not have Severity Failure.
  • Fixed integration tests.
  • Implementation of test cases for Retry functionality was fixed.
  • Retry mechanism not working while awaiting Receipt or error List to be available in MSH. 
  • AS4 Common Profile TA10 fixed: the initial implementation was translated from Minder, where it was not testing anything different from TA09 and was additionally an incorrect implementation. TA10 tests that SUT can send always the same Receipt in response to the same User Message.
  • AS4 Common Profile TA15 fixed: the initial implementation was translated from Minder, where it was not correctly implemented.

Known bugs and limitations

  • Interoperability Test Bed logs in Retry test cases need to be corrected: some Retry test cases are printing repeated and unclear error messages.
  • In test case reports of test cases using Test Action Behaviours, a better description for the Test Behaviours should be found.
  • Some log messages are showing mismatching messageId between MDC and the log message itself.


For more information, please contact us via our portal or by e-mail: EC-EDELIVERY-SUPPORT@ec.europa.eu