Development of a training strategy to bridge the big data skills gap in European official statistics

Document date: 
Wednesday, 25 April, 2018

This document outlines the findings and recommendations related to:

  • the identification of skills required for the use of big data sources,
  • the analysis of training needs from the statistical offices of ESS and
  • the definition of training objectives and content in the area of big data for official statistics.

The results from the skill analysis give a clear indication about individual big data, data science and statistical skills as well as skill groups that are currently in demand all over Europe. The report addresses skill groups at different levels of the big data Skills Framework, such as soft skills, tasks and methods (statistical and data science tasks, administrative tasks for statistical purposes, information technologies tasks for statistical purposes), tools and technologies (big data skills related to platform architecture, big data skills related to statistics and business intelligence, big data skills related to data management, big data skills related to cloud technologies, big data skills related to data mining tools, big data skills related to databases, big data skills related to Hadoop technology, big data skills related to programming languages, big data skills related to search, big data skills related to visualization technologies, upper level data science skills).

The skill analysis shows growing trends in different big data skill groups and suggests that there is a great need for big data training. There are a number of established skills and skill areas where demand is high across Europe, including Apache-related skills, Databases, Programming languages, such as Python and Javascript, Cloud technologies, Data mining etc. Other skills, such as data visualisation and a number of data management technologies are emerging trends with steady growth, but not yet to the high levels of other areas.

In order to identify big data training needs and obtain a vision about existing skills in NSIs, a survey targeted at big Data focal points in EU countries was conducted. The report describes the survey results as well as the analysis of existing skills in the ESS according to big data skills framework for official statistics.

In particular, the survey defined the groups of skills that NSIs would like to acquire:

  • Methodological skills;
  • Technical skills;
  • Visualization and storytelling skills;
  • Contextual skills and
  • Soft skills.

Several data types/data sources, such as web-scrapped data, mobile phone data, sensor data, scanner data etc. have been frequently listed as priorities.  The survey defined that training should be performed at different levels (introductory and advanced). Training can be targeted at different profiles of the employees. Individuals and big data teams can receive training. Training should take into account the big data sources and types that would be addressed in ESS.

The minimum and maximum number of trainees varies depending on the size of the NSI and the NSI training strategy.

The priorities defined in the survey included:

  • Priorities for training methods/knowledge transfer types (such as webinars, online courses along with face-to-face training);
  • Priorities for data sources/data types (webs-craped data and mobile phone data are frequently mentioned in the priorities);
  • Priorities for technologies/methods/skills (like introductory big data methodologies and skills);
  • Priorities for other issues (trainings up to 1 week, trainings on the job).

Furthermore, based on the survey results and skills required for operation with big data, a set of training objectives with corresponding content were identified. The report includes the definition of what the trainees should be able to do as a result of the training, to what standards and under what conditions. The training objectives and content defined in the report are ready to be used for the design of the effective training plan.

Finally, the report presents an analysis of the possible ways big data skills training can be provided to the staff of the statistical offices of the ESS.

Several existing tools, such as the ESTP (European Statistics Training Programme) and The European Master in Official Statistics (EMOS) are taken into consideration, as well as possible e-learning training mechanisms. In particular, options such as Webinars, MOOCs, Videolectures, Workshops and Personalized courses portals are suggested as feasible training channels.

The report highlights the necessity of using blended (face-to-face and online) training approaches in order to reach the training needs in the area of big data for official statistics.