SDMX-RI EVOLUTION

SDMX-RI evolution (SRI)

SRI’s timeline comprises three major milestones:

  1. SRI initial solution
  2. SRI intermediate solution
  3. SRI ultimate solution

“Systems must continually be adapted to the changing environment; otherwise their utility will progressively decline”.

For this reason the continuous software architecture evolution principle has been applied to the SDMX Reference Infrastructure through recurring architecture assessments.

The SRI initial solution was the starting point for implementing a first set of basic SDMX tools and modular components. However this solution did not provide sufficient performance for handling larger SDMX requests and needed further improvements.

The SRI intermediate solution’s main evolution is based on streaming, which consists in creating a lightweight load in memory, allowing a significant increase of the performances for processing SDMX requests as compared to the SRI initial solution. In addition, this solution provides support for SDMX Schema v2.0 and offers one Web Service interface: SOAP SDMX v2.0. This solution can be adopted by organisations that do not have a need to implement the SDMX 2.1 standard.

The SRI ultimate solution has been designed to follow the evolution of the SDMX standard. For this solution, a component oriented programming approach has been applied. All the existing modules and components are currently being modified to use the SDMX Common API. This API is a faithful implementation of the SDMX Information Model providing functional modules for reading, writing, querying, validating, storing, and retrieving data and metadata using SDMX.

The SRI ultimate solution provides support for SDMX Schema v2.0 & v2.1 and offers three Web Service interfaces: SOAP SDMX v2.0, SOAP SDMX v2.1 and REST SDMX v2.1.

This solution is recommended for organisations that want to follow the evolution of the SDMX standard.

SRI Ultimate Solution

In the context of the evolution of the SDMX standard, and in order to provide a scalable solution, the SRI Intermediate version will go through certain modifications which will produce what is called the “SRI Ultimate solution”. In this solution all modules will be modified to use the SDMX Common API.

The SDMX Common API is a faithful representation of the SDMX Information Model providing functional modules for reading, writing, querying, validating, storing, and retrieving data and metadata.

The benefits of the SDMX common API are listed below:

  1. It creates clearer code in the client programs by hiding the complexity of the SDMX messages,
  2. It ensures the reusability of common building block,
  3. It guarantees software interoperability,
  4. It allows different implementation of the API,
  5. It facilitates open source development
  6. It provides interfaces of methods for reading from SDMX-ML messages and writing to SDMX-ML messages.

The changes are the following:

  1. All modules migrated to the common SDMX API(6)
    1. Replaces SDMX Model/IO(5)
    2. SDMX Source implementation (7) of SDMX Common API in the context of the SRI.
  2. SRI Components APIs will be changed
    1. o due to common SDMX API which provide a set of interfaces for handling data and metadata based on SDMX Information Model
  3. Support of SDMX v2.1 messages and new query features
  4. The Web Service Provider in charge of exposing data using a WS will provide 3 interfaces for exposing
    1. SDMX v2.0 SOAP
    2. SDMX v2.1 SOAP
    3. SDMX v2.1 RESTful API

Overview of the SRI ultimate solution

Overview of the SRI ultimate solution including the extension to SDMX v2.1 interfaces

Below are presented the modifications that occurred on SRI modules for the Ultimate solution.

Controller
The controller is a module of the Web Service Provider component which interprets the SOAP/REST requests and coordinates the calls to the other SRI modules. (Common SDMX API, DR, SR)

NSI_Service_2.0
The NSI Service 2.0 is a module of the Web Service Provider component which implements the Web Service SOAP interface according to the SDMX v2.0 Web Service guidelines. This module is in charge for passing SDMX v2.0 SOAP requests, to the Controller.

NSI_Service_2.1
The NSI Service 2.1 is a module of the Web Service Provider component which implements the Web Service SOAP interface according to the SDMX v2.1 Web Service guidelines. This module is in charge for passing SDMX v2.1 SOAP requests, to the Controller.

NsiRestService
The NsiRestService is a module of the Web Service Provider component which implements the Web Service Restful API according to the SDMX v2.1 Web Service guidelines. This module is in charge for passing SDMX v2.1 Rest requests, to the Controller.

Data Retriever (3)
This module is responsible for querying the dissemination database and getting the respective “record set”, which is then streamed to the caller.

The DR is provided with the query to process and a Streaming Writer that depends on the type of the message (i.e. Generic, Compact, XS) Compared with the intermediate solution, now DR uses the common SDMX API instead of the Eurostat’s SDMX Model/IO. The features and the design are similar. However, the DR API changes due to the usage of the Common SDMX API.

Common SDMX API (6)
This is an API that provides interfaces of objects for storing data and metadata based on the SDMX information model. This API includes methods for reading and writing from SDMX-ML message and writing to SDMX-ML messages

Also, it provides interfaces of methods for reading and writing from/to SDMX-ML messages. It is the common SDMX API that is intended to be used in a inter-organisation scope in order to foster reusability of components[1].

In the context of the SRI Web Service, the following features are used:

  • Reading the SDMX-ML Data Query.
  • Reading and writing the SDMX-ML Registry Interface messages (only for the query of structures).
  • Streaming writers of the SDMX-ML Datasets.

It has replaced the Eurostat’s Model/IO from the Intermediate solution. It provides interfaces for the beans, thus the logic and the interfaces of a module is based on the interface and not on the implementation.

Dependencies between the modules of SRI Ultimate solution