Statistics Netherlands

Team Culture, Tourism and Technology

Henri Faasdreef 312

2490 HA Den Haag

The Netherlands  

(For both contact persons)

Data on the Information and Communication Technologies (ICT) usage and e-commerce in enterprises are survey data. They are collected by the National Statistical Institutes or Ministries and are in principle based on Eurostat's annual model questionnaires on ICT usage and e-commerce in enterprises.

Large part of the data collected is used to measure the progress in the implementation of one of the main political priorities of the European Commission for 2019 to 2024 – A Europe fit for the digital age. Part of this is the "European strategy for data", envisioning a single market for data to ensure the EU's global competitiveness and data sovereignty, in which context a comprehensive set of new rules for all digital services was proposed: the Digital Services Act and the Digital Markets Act, which are centrepieces of the EU digital strategy. Furthermore, the Commission and the High Representative of the Union for Foreign Affairs and Security Policy presented a new “EU cybersecurity strategy”, which is intended to bolster the EU's collective resilience against cyber threats, safeguard a global and open internet and protect EU values and the fundamental rights of its people. Furthermore, data will allow monitoring the progress towards  A Europe fit for the digital age, one of the six priorities for the period 2019-2024 of the von der Leyen European Commission.

The aim of the European survey on ICT usage and e-commerce in enterprises is to collect and disseminate harmonised and comparable information at European level.

 NACE Rev.2 2008

All economic activities in the scope of Annex I of the Commission Regulation are intended to be included in the general survey, covering enterprises with 10 or more employees and self-employed persons. These activities are: NACE Rev. 2 sections C, D, E, F, G, H, I, J, L, M and N, division 95.1.

For micro-enterprises see the sub-concepts below.

All economic activities in the scope of Annex I of the Commission Regulation are intended to be included in the general survey, covering enterprises with 2 up to 9 employees. These activities are: NACE Rev. 2 sections C, D, E, F, G, H, I, J, L, M and N, division 95.1.

The model questionnaire on ICT usage and e-commerce in enterprises provides a large variety of variables covering among others the following areas:

-          General information about ICT systems

-          Access to and use of the Internet

-          E-commerce and e-business

-          Use of cloud computing services

-          Artificial Intelligence

-          Other topics: Data utilisation, sharing, analytics and trading, Invoicing.

The annual model questionnaires and the European businesses statistics compliers’ manual for ICT usage and e-commerce in enterprises comprise definitions and explanations regarding the topics of the survey.

The statistical unit used is the business enterprise.

Target Population

As required by Annex of the Commission Implementing Regulation, enterprises with 10 or more employees and self-employed persons shall be covered by the survey.

For micro-enterprises see the sub-concepts below.

[2-9]

The geographic scope of the ICT-survey is the (European) Netherlands, i.e. excluding Caribbean Netherlands (which are also part of the Kingdom of the Netherlands).

Data for a specific set of variables were delivered on NUTS 2 regional level.

Years 2022 and 2023.

Not applicable

Percentages of enterprises, Percentages of turnover, Percentages of employees and self-employed persons, Million euro.

For the 2023 ENT survey, the reference period is the current survey period (2023) except for the modules on e-commerce and invoicing where respondent should report their 2022 values.

Complementary national legislation constituting the legal basis for the survey on the use of ICT in enterprises:

The Ministry of Economic Affairs uses continuously the outcomes of the ICT-survey to support governmental decision making. See for example the publications on the Government Official Journal on the ICT-related modules van "Programma Digital Europe" and on the "Act to Promote Enterpreneurial Digital Resilience".

Not available

Regulation (EC) No 223/2009 on European statistics (recital 24 and Article 20(4)) of 11 March 2009 (OJ L 87, p. 164), stipulates the need to establish common principles and guidelines ensuring the confidentiality of data used for the production of European statistics and the access to those confidential data with due account for technical developments and the requirements of users in a democratic society.

At national level : 

At National level, the Dutch Data Protection Authority (DPA) supervises compliance with regulations of the law concerning personal data protection. An important contact person for the Dutch DPA is the CBS Data Protection Officer. Statistics Netherlands has its own Data Protection Officer (DPO). This officer monitors the implementation of and compliance with the GDPR by CBS and keeps a register of all personal data processing.

 The measures of Statistics Netherlands (CBS) to safeguard the privacy and confidentiality of the collected data include technical and logistical measures:

 - All CBS employees are bound by an obligation of confidentiality and have signed a confidentiality agreement.

- CBS only uses data for statistical and scientific purposes. CBS is excluded by law from using the data for fiscal, administrative, verification and legal purposes. Furthermore, CBS is not allowed to use data for marketing.

- All statistical processes at CBS are certified in terms of personal data protection. This privacy proof audit is carried out by an accredited external party.

Data are transmitted via eDamis (encrypted) and delivered to a secure environment where they are treated. Flags are added for confidentiality in case results must not be disclosed.

At national level : 

More specific measures of Statistics Netherlands (CBS) to safeguard the privacy and confidentiality of the collected data are: 

- When respondents complete a survey or submit any data, this information is delivered to CBS in encoded form. The data are received by CBS in a secured environment. Only authorised personnel shall have access to these data.

 - At the earliest possible stage in the process, all directly identifiable personal data are removed from the files. This means datasets for research will never contain any data such as names, addresses or citizen service numbers.

Additional information about confidentiality policies and measures are available via Statistics Netherlands webpagina and brochures.

There is no formal publication planning for the statistical outputs in StatLine. Each Friday, CBS publishes the publication planning for its upcoming press releases (in Dutch/in English). The long-term planning is provisional. The target is to release the statistical output van de ICT survey within the survey year. 

 https://www.cbs.nl/en-gb/publication-calendar

The general data release policy of Statistics Netherlands (CBS) is available here.

This policy is in line with the Community legal framework and the European Statistics Code of Practice and the Eurostat protocol on impartial access to Eurostat data for users.

Annual

Annual news releases are available online in December of the survey year or January of the following year.

The National dissemination of results is carried out twice a year:

- At the beginning of the data collection. Enterprises are provided the links to the statistical outcomes of the previous year along with the invitation to participate in the survey "ICT usage in enterprises" (in Dutch).

 - At the publication and release of the outcomes of the current year via Statline and MKB Statline. The latter is an open data site dedicated to statistics on Small and Medium-sized Enterprises (SMEs).

Furthermore, the yearly dissemination of results includes also:

- The publication "ICT, knowledge and economy" (in Dutch) is available as a web publication and it is free of charge. (The release 2023 is based on data collected in the 2022 survey.) A link to the publication series on "ICT, knowledge and economy" is found here.

- The publication "Cybersecurity Monitor" (in Dutch) is available as a PDF file and it is also free of charge. (The release 2022 is based on data collected in the 2022 survey.) A link to the publication series on cybersecurity is found here.

See detailed section 10.3.1.

Results for selected variables collected in the framework of this survey are available for all participating countries on Digital economy and society of Eurostat website.

At national level :

Outcome tables are available via de websites Statline and MKB Statline (dedicated website for SME's). The number of accesses is not available.

Microdata requests are granted under a number of conditions. See Microdata.

Not requested.

Not requested

Methodological explanation about the ICT survey in enterprises is provided here.

Not requested

The quality asessment of the statistical output of the survey on ICT usage in enterprise is periodically executed and published in the report series ICT, knowledge and economy (in Dutch). The most recent figures on the use of ICT usage in enterprises are hierin evaluated and discussed in detail. Figures are often placed in an international perspective to compare the Netherlands with other countries.

The European businesses statistics compliers’ manual for ICT usage and e-commerce in enterprises provides guidelines and standards for the implementation of the surveys. It is updated every year according to the changed contents of the model questionnaires.

At national level :

The general quality management principles in Statistics Netherlands are actively promoted by means of the Statistical Netherlands’ Quality Assurance Framework at Process Level (Quality Guidelines for Process Assurance). These guidelines are continuously revised and operationalized in a template to support National Information Protection. The template includes general information of the statistical (production) process as well as workflows and descriptions at different detail levels (top-down). Moreover, documentation on connected information systems and software are also provided along with self-assessments and GDPR-documentation. Monitoring of compliance and audits are carried out by the department of Methodology of Statistics Netherlands. The last update to these documentation about the process ICT-usage in enterprises was executed in 2022.

European level :

At European level, the recommended use of the annual Eurostat model questionnaire aims at improving comparability of the results among the countries that conduct the survey on ICT usage and e-commerce in enterprises. Moreover, the Methodological Manual provides guidelines and clarifications for the implementation of the surveys.

National level :

Production process is designed to be fully automatized from reading the survey data to the ready-to-upload data delivery. Evaluations and consistency tests are rule-driven. Process includes also rest-points to check data and metadata quality. Handlings are minimal and reproducibility of results is increased. Scripts (and packages) are brought under a version control and management system (Git). 

European level : 

At European level, European Commission users (e.g. DG CNECT, DG GROW, DG JUST, DG REGIO, DG JRC) are the principal users of the data on ICT usage and e-commerce in enterprises and contribute in identifying/defining the topics to be covered. Hence, main users are consulted regularly (at hearings, task forces, ad hoc meetings) for their needs and are involved in the process of the development of the model questionnaires at a very early stage.

User needs are considered throughout the whole discussion process of the model questionnaires aiming at providing relevant statistical data for monitoring and benchmarking of European policies.

National level :

The main user of the data on ICT usage and e-commerce in enterprises is the Dutch Ministry of Economic Affairs (EZK). Other important users are the National Cyber Security Centrum (NCSC), Digital Trust Center (DTC), Netherlands Enterprise Agency (RVO) and research organizations like TNO. 

European level : 

At European level, contacts within the Commission, the OECD and other stakeholders give a clear picture about the key users' satisfaction as to the following data quality aspects: accuracy and reliability of results, timeliness, satisfactory accessibility, clarity and comparability over time and between countries, completeness and relevance. Overall users have evaluated positively (good, very good) the data quality on the ICT usage and e-commerce in enterprises.

National level :

The current level of disaggregation of enterprises' size classes reaches the requirements of the main users of the data of the ICT usage and e-commerce in enterprises.

As for the respondents, in 2023 the survey has been chosen by the Methodology Department to participate in a pilot project to measure the enquiry load/pressure experienced by respondents of enterprises participating in the ICT-survey. One of the indicators is the length of time used to fill-in the survey. Respondents are also requested to provide a score and feedback on the survey content and usefulness. 

Detailed information is available in “ Annex I _ Completeness “ excel file - related to questionnaire, coverage, additional questions.

Not requested. 

Comments on reliability and representativeness of results and completeness of dataset

These comments reflect overall standard errors reported for the indicators and breakdowns in section 13.2.1 (Sampling error - indicators) and the rest of the breakdowns for national and European aggregates, as well as other accuracy measurements. The estimated standard error should not exceed 2pp for the overall proportions and should not exceed 5pp for the proportions related to the different subgroups of the population (for those NACE aggregates for the calculation and dissemination of national aggregates). If problems were found, these could have implications for future surveys (e.g. need to improve sampling design, to increase sample sizes, to increase the response rates).

More detailed information is available in “ Annex II. _ Accuracy “ excel file - related to European aggregates, comments on reliability and use of flag.

For calculation of the standard error see 13.2.1.1.

Standard error (for selected indicators and breakdowns)

Precision measures related to variability due to sampling, unit non-response (the size of the subset of respondents is smaller than the size of the original sample) and other (imputation for item non-response, calibration etc.) are not (yet) required from the Member states for all indicators.  Eurostat will make basic assumptions to compute these measures for all indicators produced (e.g. stratified random sampling assuming as strata the crossing of the variables “Number of employees and self-employed persons” and “Economic Activity” as it was defined in the 3 tables of section 18.1).

More detailed information is available in“ Sample and standard error tables 2023 “ excel file – worksheets starting with “Standard error".

Calculation of the standard error

Various methods can be used for the calculation of the standard error for an estimated proportion. The aim is to incorporate into the standard error the sampling variability but also variability due to unit non-response, item non-response (imputation), calibration etc. In case of census / take-all strata, the aim is to calculate the standard errors comprising the variability due to unit non-response and item non-response.

See detailed sections below.

See concept 18.1.1. A) Description of  frame population.

See 18.1.1. A) Known shortcomings of frame population

The over-coverage of enterprises with 10+ employees is 2.1%. Respondents argue that their economic activity requires a minimum ICT usage (small construction enterprises  or shops having a physical store only) and therefore the questions are either not applicable for the enterprise or they cannot answer the questions and require to hire a bookkeeper. Last, there were few enterprises reporting bankruptancy or declared to have ceased their operations.

The over-coverage of enterprises with 2-9 employees is 4.3%. Respondents reply often that either their economic activity does not require ICT usage (as house moving enterprises, carpenters, plasterworkers or street marketers/peddlers) or that their size class is misspecified e.g. enterprise receives an enquiry for small enterprises while they are actually self-employees. Last, there were also some enterprises reporting bankruptancy or declared  to have ceased their operations.

Not requested

We did not have measurement errors.

See detailed sections below.

See detailed sub-concepts below.

The following table contains the number of units (i.e. enterprises), by type of response to the survey and by the percentage of these values in relation to the gross sample size.

After the invitation letters in February, we have 3 reminders in total until august in order to convince companies to fill in the survey.

Not available.

The items of the question B4 (What was the percentage breakdown of the value of web sales in 2022 by type of customer i.e. B4a) on B2C and B4b) on B2B plus B2G)) have an item non-response rate of 25%. (The item-response rate is 75%.)

The items of the question E2 (On the Purpose of Use of Artificial Intelligence software or system) i.e. variables E_AI_PMS, E_AI_PPP, E_AI_PBAM, E_AI_PLOG, E_AI_PITS, E_AI_PFIN and E_AI_PRDI) have an item non-response rate of 21% for variables . (The item-response rate is 79%.)

 The items of the optional question A8 (On the use of social media) i.e. variables E_SM_PCUDEV, E_SM_PBPCOLL, E_SM_PRCR and E_SM_PEXCHVOK have an item non-response rate of 15% for variables. (The item-response rate is 85%.)

The item of the optional question F2.bis (On the percentage of e-invoices sent -in electronic or paper form- to private customers, other enterprises or public authorities), i.e. variable E_INV4S_AP has a high item non-response rate of 64%. (The item-response rate is 36%.) 

Please, for a further explanation on these non-response rates see the comments of section 19.

Other comments relating to the item non-response

We did not have processing errors

Not requested

See detailed section below.

Not applicable

European level : 

Data are to be delivered to Eurostat in the fourth quarter of the reference year (due date for the finalised dataset is 5th October). European results are released before the end of the survey year or in the beginning of the year following the survey year (T=reference year, T+0 for indicators referring to the current year, T+12 months for other indicators referring to the previous year e.g. e-commerce).

At national level : 

The number of months between the reference period and the publication date are 12.

 See detailed section below.

The first data delivery was sent to Eurostat on 05-10-2022.

The model questionnaire is generally used by the countries that conduct the survey on ICT usage and e-commerce in enterprises. Due to (small) differences in translation, in the used survey vehicle, in non-response treatment or different routing through the questionnaire, some results for some countries may be of reduced comparability. In these cases, notes are added in the data.

Detailed information on differences in the wording of the questions in the national questionnaires is available in “ Annex I _ Completeness “ excel file - related to questionnaire, coverage, additional questions.

Data for specific set of variables were delivered on NUTS 2 regional level. There is no problem of comparability across the country’s regions.

Not applicable

See section below.

The length of comparable time series depends on the module and the variable considered within each survey module. Additional information is available in annexes attached to the European metadata.

There are no changes with regard to last year, hence we do not expect differences inherent to this issue.

Not applicable

 Not applicable

Not applicable

Not applicable

No revisions are planned. Data have a final status from the first time they are published.

In case we need to correct a value, we republish the tables on Statline along with an explanation on what has been changed.

 Not requested

A) Frame population description and distribution

For more information see concept 18.1.1.

B) Sampling design - Sampling method

 This section included a description of the sampling method used (e.g. stratified random sample, quota sampling, cluster sampling; one-stage or two-stage sampling) and information which variables were used to stratify, the categories of those variables, in particular for the NACE Rev. 2 categories related to the "possible calculation of European aggregates", and the final number of strata: 

 Enterprises with 100 or more employees are all included in the gross sample. For enterprises with 2-100 employees a random sample is used, stratified by NACE (32) and size class (6). Moreover, sample size is also based on prior experiences and cost considerations. For the optimal distribution of sample units across the strata, the Neymann allocation method is used. Enterprises that are drawn in other business survey samples have a reduced likelihood of being drawn into the sample of this survey.

C) Gross sample distribution

More detailed information is available in “ Sample and standard error tables 2023  “ excel file (Worksheet: GROSS SAMPLE)

D) Net sample distribution

More detailed information is available in “ Sample and standard error tables 2023  “ excel file (Worksheet: NET SAMPLE)

A) Description of frame population

 B) Frame population distribution

More detailed information is available in “ Sample and standard error tables 2023  “ excel file (Worksheet: FRAME POPULATION)

Annual

See detailed sections below.

Web survey only.

We have used the validation tool provided by Eurostat (both Acceptance and Production environments) plus an in-house developed tool for a year to year comparison.

Grossing-up procedures

Depending on the variable that has to be reported, background variables such as “number of persons employed”, "number of enterprises" and "turnover" are needed for grossing up the responses from industry level towards economic sector level. Most variables collected are qualitative ones. A few are quantitative variables, i.e. percentage or real values.

Three variable cases are considered:

Case 1. Variables such as (overall) percentage of persons employed having access remotely to IT-services of their enterprises,
Case 2. Variables such as (overall) percentage of enterprises using social media or having a website, or
Case 3. Variables such as (overall) percentage of turnover via online platforms.

We use weights to aggregate from industry level towards economic sector level. Subgroup weights (quotients) are based on the number of enterprises that are within the strata (i.e. survey size classes by sector levels) of the target population and the corresponding number of enterprises in the response. These weighting factors (partially) correct biases caused by the unit non-response for target variables of Case 1 and Case 2. The e-commerce turnover-related variables are estimated using weighting factors based on enterprises' turnover (variables of Case 3) and are less robust to unit non-response.

This value varies from breakdown to breakdown and question to question. We report the values in our logging files, but don't report them in a table, so it is hard to give a number.

Not applicable

Not applicable

On the ICT security module, enterprises belonging to overseas concerns and franchise entrepreneurs reply frequently that ICT security and auditing is administered or supported from their head-offices. In those cases, the administrative clerk or enterpreneur who handles the local office operations adduce to fill-in this module to the best of his/her knowledge.

The statistical quality of the modules on e-commerce and invoices relies on the effort and commitment of the respondents at the financial department of enterprises. They report that questions are hard to answer and very burdesome. The early respondents (February till May) remark that their enterprises' annual reports are not yet available and cannot provide official revenues and sales values. 

Large enterprises report also that the survey questionnaire requires to be filled out by more than one person within the enterprise. ICT managers (the original target respondents of this survey) remark that modules on 'E-commerce' and 'Invoices' are either left empty or with '0' as the ICT department cannot provide financial valuees and neither information on martketing or R&D domains i.e. the submodules over 'Use of social media', 'Other use of the internet' and purposes of 'Artificial Intelligence'.