STRUCTURAL VALIDATION PROCESS

A nominal structural validation process can be described by a number of steps.

Note: It is implied that the implementation of the SDMX standard has been completed for the respective statistical production domain. DSDs required for service execution are available in SDMX registries (e.g. Euro SDMX Registry, Global SDMX Registry).

  1. The Data Provider inputs the SDMX IM-compliant dataset through EDAMIS. The IS4STAT process manager is notified of the incoming files and initiates the validation process by calling the STRUVAL service.

  2. STRUVAL verifies that the associated DSD document is a valid, self-contained xml structure containing all code lists, concepts and key families necessary for the validation of the specific dataset.

  3. STRUVAL verifies that the input dataset is a well-formed xml structure (e.g. no xml syntax errors, mismatched open/closed tags, illegal characters).

  4. STRUVAL checks the values of all concepts against the concept data types, code lists and applicable constraints. Errors detected may include:

    1. Completeness errors:

      • Missing mandatory data items (dimensions, data elements or attributes)

      • Invalid data items (unexpected or undeclared dimensions, concepts or attributes)

      • Duplicated series / observations

    2. Value errors:

      • Invalid data formats or empty strings

      • Invalid codes

      • Invalid numerical (observation) values

    3. Constraint violations:

      • For content constraints, the tuple of the dimension concept values falls out of the allowed cube region (or falls in an explicitly excluded cube region).

      • For attachment constraints, the required metadata from the metadata target region is missing.

    4. Consistency errors, e.g.:

      • Periods do not correspond to frequencies

      • If EMBARGO_DATE is set, check that CONF_STATUS is "N"

      • SENDER_ID field from the header is inconsistent with REF_AREA

  5. If the input dataset is structurally valid, it is sent by the IS4STAT process manager to the next phase (either Content validation, format conversion or directly to the production system). The data provider receives the STRUVAL validation report through EDAMIS.

  6. If the input dataset has not a valid structure (ie, it fails one of the check 3 and 4 above), it is not further processed. The Data provider receives the STRUVAL validation report through EDAMIS.

  7. If the STRUVAL validation cannot be executed, the domain manager is notified, through email or IS4STAT Input Hall client) with the detail of the technical error.