WP3 – Implementing the Business Architecture
The Business Architecture for Data Validation in the ESS presents three different scenarios for integration of a service-based validation procedure into the interface between national statistical production and delivery of data to Eurostat. As part of this project we built pilot implementations of these scenarios and integrated them in our own production systems.
To test these implementations we agreed upon a set of validation rules in VTL and applied them to test data. We built test configurations out of existing services and some components that we specifically implemented for this project. The purpose of these pilots was to give us some idea of the effort necessary to build a production-ready system.
The first scenario depends on using the national infrastructure to run validation rules agreed upon in the ESS. A necessary prerequisite for this method is the translation of the validation rules and the data structures these rules are based on into representations that the national systems can understand. Towards that end, we developed a proof-of-concept translator from VTL to T-SQL, the language used for data validation in Poland.
The second scenario envisions a validation process orchestrated by the NSO that makes use of services provided by Eurostat or other members of the ESS. The original plan was for Sweden to try and build a production chain that uses shared STRUVAL service and a replicated CONVAL. Since the VTL-based CONVAL service will not be available in time for this ESSnet, we are currently working on a replacement solution for our pilot implementation.
In the final scenario, the NSO simply provides the input data, while the validation procedure itself is performed using shared services on the ESS platform. The result will be a validation report provided to the NSO, on the basis of which data editing can be performed if necessary. Portugal is currently working on a solution that will represent this method using shared services provided by Eurostat.
The pilot implementations of the three scenarios were just a means to an end. The main goal of work package 3 was to perform a cost-benefit analysis of implementing these methods in order to support other NSOs in their decision on which path to follow. We prepared an evaluation of these implementations and provide a system that allows decision makers to weigh different cost drivers against each other and include the results in their considerations.