Processing open source data with exchangeable components

Interoperable Meta Data and Processing Components

2018.02 Interoperable Meta Data and Processing Components for Open Source Information Mining

When is this action of interest to you? 

You are a public administrator and your authority wants to derive information from open source data for informative or investigative purposes.

What is this action about?

Many authorities need to process open source data from the internet and other sources, such as news articles, mapping data, open government data, etc. for purposes of their mission. The data needs to be processed by passing it through a number of software components (for example a component to do automatic language translation) before a human can effectively use it. For various processing steps software tools (commercial and non-commercial) already exist and are in use by authorities of EU Member States and EU institutions. However, it proves difficult to exchange and reuse tools between authorities since often they were developed only with the interoperability within an individual authority or country in mind. This leads to costly duplication of effort and hampers collaboration between authorities across different countries. To tackle this issue a set of best practices needs to be establised which defines minimum interoperability standards (based on previous standardisation and industry standards). A component that adheres to these standards can then more easily be reused for different tasks by multiple authorities across Europe. 

What are the objectives?

The objective is to define a set of best practice standards to make software components in the field of open source data processing exchangeable which allows cost-saving and less duplicated effort.

The best practice standards comprise mainly of: 

  • Standards for Meta Data describing the processed data
  • Guidelines on how to develop new and adapt existing components for better interoperability

Where possible, existing standards and industry best practices will be taken into account.

Furthermore, the project aims to develop a set of interchangeable components and a composited demonstrator application.

What are the benefits?

For citizens

  • Faster service from authorities which are equipped with the latest technology

For EU institutions and other EU bodies

  • Cost-savings because open source data processing components can be reused across institutions
  • Faster reaction to changing requirements, since software can be reused and needs not to be developed from scratch

For Member States

  • Cost-savings because software components from other EU Member States can be adopted
  • Easier collaboration across authorities and countries since a best-practice approach is adopted 
  • Adoption of a proven best-practice approach for developing new software components

What are the next steps?

In 2018 the focus will be to work with project stakeholders to understand which standards exist and can be adopted for our best practice set of standards. Furthermore, a list of already used processing components will be created to better understand how they could be brought together in a composite application.