Content validation - SDMX InfoSpace


The Content Validation service (called CONVAL) is currently in development. Its purpose will be to perform the validation of the content of statistical datasets based on validation rules and constraints formulated by the respective domain managers. It will be able to carry out the following key checks on a data file:

  1. Basic logical checks, verifying expected relationships and consistency between cells within the same file.

  2. Basic content checks, verifying if observation values expected are present and data types are correct. Checks may include detection of:

    1. missing or unexpected series

    2. hole in time series

    3. zero values

    4. negative values

    5. fixed range checks.

  3. General plausibility and consistency checks, performing vertical and horizontal validations within the dataset. Checks may include inspection of

    1. additivity of breakdowns

    2. outliers

    3. consistency between values (e.g. prices)

    4. unadjusted and adjusted series

  4. Advanced plausibility and consistency checks perform validations between multiple correlated datasets. As of Q4 2016 the service only covers time series checks from this category. Inclusion of further operations are subject to future developments.

The CONVAL Service accepts one or more datasets of known structure and a description of the validation task to be performed, and returns a validation report.

The validation task to be performed is normally specified using a well-known (i.e. standardized) logical name of the validation task, which corresponds to a validation program prepared in advance by the domain experts. This logical name is typically fixed by the domain manager, or stored in a registry for user reference.

With the release 1.1 of the standard Validation and Transformation Language (VTL) in the near future it is planned that the CONVAL Service will be able to accept a specification of the validation task; this will be in the form of a directly provided VTL program which is then executed by the service. Such VTL programs may be provided on an ad-hoc basis by the service user, or retrieved from a repository of standard validation programs.

For more information, have a look at what a nominal content validation process involves.