National Bureau of Statistics

Department of education, culture and information society

Branimirova 19, 10000 Zagreb, Croatia

Data on the Information and Communication Technologies (ICT) usage and e-commerce in enterprises are survey data. They are collected by the National Statistical Institutes or Ministries and are in principle based on Eurostat's annual model questionnaires on ICT usage and e-commerce in enterprises.

Large part of the data collected is used to measure the progress in the implementation of one of the main political priorities of the European Commission for 2019 to 2024 – A Europe fit for the digital age. Part of this is the "European strategy for data", envisioning a single market for data to ensure the EU's global competitiveness and data sovereignty, in which context a comprehensive set of new rules for all digital services was proposed: the Digital Services Act and the Digital Markets Act, which are centrepieces of the EU digital strategy. Furthermore, the Commission and the High Representative of the Union for Foreign Affairs and Security Policy presented a new “EU cybersecurity strategy”, which is intended to bolster the EU's collective resilience against cyber threats, safeguard a global and open internet and protect EU values and the fundamental rights of its people. Furthermore, data will allow monitoring the progress towards  A Europe fit for the digital age, one of the six priorities for the period 2019-2024 of the von der Leyen European Commission.

The aim of the European survey on ICT usage and e-commerce in enterprises is to collect and disseminate harmonised and comparable information at European level.

 NACE Rev.2 2008

All economic activities in the scope of Annex I of the Commission Regulation are intended to be included in the general survey, covering enterprises with 10 or more employees and self-employed persons. These activities are: NACE Rev. 2 sections C, D, E, F, G, H, I, J, L, M and N, division 95.1.

For micro-enterprises see the sub-concepts below.

Micro-enterprises are not in the scope of the survey.

The model questionnaire on ICT usage and e-commerce in enterprises provides a large variety of variables covering among others the following areas:

-          Access to and use of the Internet

-          E-commerce and e-business

-          Use of cloud computing services

-          Artificial Intelligence

-          Other topics: Data utilisation, sharing, analytics and trading, Invoicing.

The annual model questionnaires and the European businesses statistics compliers’ manual for ICT usage and e-commerce in enterprises comprise definitions and explanations regarding the topics of the survey.

Enterprise

Target Population

As required by Annex of the Commission Implementing Regulation, enterprises with 10 or more employees and self-employed persons shall be covered by the survey.

For micro-enterprises see the sub-concepts below.

Not applicable.

Whole territory of the country is covered. Data for a specific set of variables were delivered on NUTS 2 regional level.

Years 2023 and 2024.

Not applicable

Percentages of enterprises, Percentages of turnover, Percentages of employees and self-employed persons, Million euro.

Referenced year is 2023, except for e-commerce questions which are referenced to 2022.

Complementary national legislation constituting the legal basis for the survey on the use of ICT in enterprises:

National law on Statistics, 2020

Program of statistical activities of the Republic of Croatia 2021-2027, 2023

Data sharing procedure is established with International Telecommunication Union.

Regulation (EC) No 223/2009 on European statistics (recital 24 and Article 20(4)) of 11 March 2009 (OJ L 87, p. 164), stipulates the need to establish common principles and guidelines ensuring the confidentiality of data used for the production of European statistics and the access to those confidential data with due account for technical developments and the requirements of users in a democratic society.

At national level : 

Confidentiality protocol is the one used in SBS survey. If there are less than 3 enterprises within particular aggregate, data is considered confidential.

Data are transmitted via eDamis (encrypted) and delivered to a secure environment where they are treated. Flags are added for confidentiality in case results must not be disclosed.

At national level : 

If there are less than 3 enterprises within particular aggregate, data is considered confidential.

There is a release calendar and is publicly available at CBS website.

Release calendar is publicly available at CBS website.

Not available.

Annual

Not available.

Publication on ICT data for 2023 will be released on 06/12/2023. 

See detailed section 10.3.1.

Results for selected variables collected in the framework of this survey are available for all participating countries on Digital economy and society of Eurostat website.

At national level :

Not available.

Not available.

Not requested

Not requested

Not publicly available.

Not requested

Not publicly available.

The European businesses statistics compliers’ manual for ICT usage and e-commerce in enterprises provides guidelines and standards for the implementation of the surveys. It is updated every year according to the changed contents of the model questionnaires.

At national level :

The Methodological Manual provides guidelines and standards for the implementation of the surveys in the Member States. It is updated every year according to the changed contents of the model questionnaires. The methodological manual is provided to CBS staff before survey conduct. Quality assurance procedures described in manual are strictly followed through training courses and use of best practices.

European level :

At European level, the recommended use of the annual Eurostat model questionnaire aims at improving comparability of the results among the countries that conduct the survey on ICT usage and e-commerce in enterprises. Moreover, the Methodological Manual provides guidelines and clarifications for the implementation of the surveys.

National level :

Quality management is currently work in progress. There are several ongoing projects through which quality management system will be developed. Pilot programme is completed resulting in development of quality indicators database for different statistical domains within NSI. Goal is to link several different sources of data to create unified data source for quality indicators and metadata information.

European level : 

At European level, European Commission users (e.g. DG CNECT, DG GROW, DG JUST, DG REGIO, DG JRC) are the principal users of the data on ICT usage and e-commerce in enterprises and contribute in identifying/defining the topics to be covered. Hence, main users are consulted regularly (at hearings, task forces, ad hoc meetings) for their needs and are involved in the process of the development of the model questionnaires at a very early stage.

User needs are considered throughout the whole discussion process of the model questionnaires aiming at providing relevant statistical data for monitoring and benchmarking of European policies.

National level :

Central agency for Development of Digital Society

Ministry of Regional Development and EU funds

Researchers

European level : 

At European level, contacts within the Commission, the OECD and other stakeholders give a clear picture about the key users' satisfaction as to the following data quality aspects: accuracy and reliability of results, timeliness, satisfactory accessibility, clarity and comparability over time and between countries, completeness and relevance. Overall users have evaluated positively (good, very good) the data quality on the ICT usage and e-commerce in enterprises.

National level :

At present moment, there are no tools to measure user satisfaction.

Detailed information is available in “ Annex I _ Completeness “ excel file - related to questionnaire, coverage, additional questions.

Not requested. 

Comments on reliability and representativeness of results and completeness of dataset

These comments reflect overall standard errors reported for the indicators and breakdowns in section 13.2.1 (Sampling error - indicators) and the rest of the breakdowns for national and European aggregates, as well as other accuracy measurements. The estimated standard error should not exceed 2pp for the overall proportions and should not exceed 5pp for the proportions related to the different subgroups of the population (for those NACE aggregates for the calculation and dissemination of national aggregates). If problems were found, these could have implications for future surveys (e.g. need to improve sampling design, to increase sample sizes, to increase the response rates).

More detailed information is available in “ Annex II. _ Accuracy “ excel file - related to European aggregates, comments on reliability and use of flag.

For calculation of the standard error see 13.2.1.1.

Standard error (for selected indicators and breakdowns)

Precision measures related to variability due to sampling, unit non-response (the size of the subset of respondents is smaller than the size of the original sample) and other (imputation for item non-response, calibration etc.) are not (yet) required from the Member states for all indicators.  Eurostat will make basic assumptions to compute these measures for all indicators produced (e.g. stratified random sampling assuming as strata the crossing of the variables “Number of employees and self-employed persons” and “Economic Activity” as it was defined in the 3 tables of section 18.1).

More detailed information is available in“ Sample and standard error tables 2023 “ excel file – worksheets starting with “Standard error".

Calculation of the standard error

Various methods can be used for the calculation of the standard error for an estimated proportion. The aim is to incorporate into the standard error the sampling variability but also variability due to unit non-response, item non-response (imputation), calibration etc. In case of census / take-all strata, the aim is to calculate the standard errors comprising the variability due to unit non-response and item non-response.

See detailed sections below.

See concept 18.1.1. A) Description of  frame population.

Over-coverage rate is 0,8%.

Not requested

No errors were found.

See detailed sections below.

See detailed sub-concepts below.

The following table contains the number of units (i.e. enterprises), by type of response to the survey and by the percentage of these values in relation to the gross sample size.

Advance notifications were sent to all enterprises by e-mail and by postal service before the start of the data collection. Reminders were sent by e-mail to enterprises which did not complete the questionnaire. Reminders were sent 4 times. Non-responders were also tried to be contacted by telephone.

Not applicable.

Item non-response rate was 0,9%.

Other comments relating to the item non-response

No errors were found.

Not requested

See detailed section below.

Not applicable

European level : 

Data are to be delivered to Eurostat in the fourth quarter of the reference year (due date for the finalised dataset is 5th October). European results are released before the end of the survey year or in the beginning of the year following the survey year (T=reference year, T+0 for indicators referring to the current year, T+12 months for other indicators referring to the previous year e.g. e-commerce).

At national level : 

Data are to be delivered to Eurostat in the fourth quarter of the reference year (due date for the finalised dataset is 5th October). European results are released before the end of the survey year or in the beginning of the year following the survey year (T=reference year, T+0 for indicators referring to the current year, T+10 months for other indicators referring to the previous year e.g. e-commerce).

 See detailed section below.

Data was delivered on 9th October.

The model questionnaire is generally used by the countries that conduct the survey on ICT usage and e-commerce in enterprises. Due to (small) differences in translation, in the used survey vehicle, in non-response treatment or different routing through the questionnaire, some results for some countries may be of reduced comparability. In these cases, notes are added in the data.

Detailed information on differences in the wording of the questions in the national questionnaires is available in “ Annex I _ Completeness “ excel file - related to questionnaire, coverage, additional questions.

Data for specific set of variables were delivered on NUTS 2 regional level. There is no problem of comparability across the country’s regions.

Not applicable

See section below.

The length of comparable time series depends on the module and the variable considered within each survey module. Additional information is available in annexes attached to the European metadata.

There were no changes in the survey since implementation of SU enterprise.

Not applicable

 Not applicable

Not applicable

Not applicable

Although the CBS employees make maximum efforts to reduce errors prior to the data release, some might occur occasionally. It is important to preserve the confidence in the official statistics, as well as in the CBS as the institution whose highest goal is dissemination of official statistics. For that reason, it is crucial that the CBS acknowledges and documents the error, and that a candid and professional explanation is given.

Given that there is a high risk of users retrieving and using the data before errors have been detected and corrected, the errors must be published in the shortest period possible, and users must be enabled to see clearly what has been corrected.

Having all of this in mind, corrections of material errors in the CBS statistics shall always be indicated with a note pointing out the amendments and that the incorrect data released must not in any case be replaced with the revised version without providing prior access to both versions. This note must never be removed. If errors occur in larger publications (this includes all except First Releases), the CBS is obliged to issue a list of all required corrections (corrigenda) which clearly indicates which publication has an error and where exactly the amends have been made: on which pages of the original publication and in which tables. The corrigenda are published in an electronic format together with the publication. If they are available on the CBS website, they will be accompanied by a notice stressing it being a case of a revised version, and the release date of the corrections. Printed corrigenda are inserted into the printed publications available for sale, and to all copies of publications available for use in the CBS library.

Procedures for handling errors of the released data are adapted to the gravity of error. Excluding proofing/printing errors, the basic procedure following detection of an error is as follows:

1. a correction of a detected error, and reissuing the data/publications or corrigenda only in case of a larger publication on the CBS website in shortest period of time possible

2. a notice stating the case of an amendment next to the link with the revised data or a notice stating availability of the corrigenda in cases of larger publications together with the release data of amended data

3. an e-mail to the subscribers of the data/publications in which an error has been detected containing electronic version or the revised data or corrigenda in cases of larger publications, together with an apology letter and explanation as to why the error/errors occurred

4. sending printed version of the revised data to the subscribers, regardless of whether it was the case of data processing, purchase of publications, or just the corrigenda, if it is a larger publication together with an apology letter and explanation of the error 

In the case of a serious error that requires a period longer than one working day to correct, the incorrect data/publication shall be removed from the CBS website, and a message stating when the corrected version or corrigenda will be posted on the homepage. With the release of the amended version of the data/publication or corrigenda, the first released version is released simultaneously, in which the amendment is displayed clearly. In other words, released data containing an error shall never be replaced by the correct version, without being clearly
displayed.

If an error is detected and corrected on the day of the release, the correction notice must state precise release time of the correction. Corrections of proofing/printing errors shall be made without issuing a correction notice. The errors that are discovered in released publications must be conveyed to the relevant statistics department without any delay, to correct the error in the shortest period possible. Apart from the statistics department in charge of the statistics, the Information, Services and Publications Directorate must be involved as well in all stages of preparation and issuing of the amendment or corrigenda. The employees of the specialist department in charge of the statistical area in which the error was detected, must, first of all, focus on correcting the detected error, while the ISPD employees are in charge of giving a notice to internal and external users and informing the Director General of the CBS.

 Not requested

A) Frame population description and distribution

For more information see concept 18.1.1.

B) Sampling design - Sampling method

Description of the sampling method used (e.g. stratified random sample, quota sampling, cluster sampling; one-stage or two-stage sampling) and information which variables were used to stratify, the categories of those variables, in particular for the NACE Rev. 2 categories related to the "possible calculation of European aggregates", and the final number of strata: 

Stratified random sampling was used for small enterprises (with 10-49 employees), NACE Rev. 2 economic activity was used as stratification variable (25 strata in total). In each particular stratum we used random sample selection of units. If allocated number of units is lower than 8, 8 units are allocated to the sample. Census approach was used for medium and large enterprises (50 and more employees). None of the procedures for the coordination or non-overlapping with samples of other surveys were applied. Sample rate by number of enterprises is 32%, sample rate by turnover value is 83%, while sample rate by number of persons employed is 82%.

C) Gross sample distribution

More detailed information is available in “ Sample and standard error tables 2023  “ excel file (Worksheet: GROSS SAMPLE)

D) Net sample distribution

More detailed information is available in “ Sample and standard error tables 2023  “ excel file (Worksheet: NET SAMPLE)

A) Description of frame population

 B) Frame population distribution

More detailed information is available in “ Sample and standard error tables 2023  “ excel file (Worksheet: FRAME POPULATION)

Annual

See detailed sections below.

Web survey - introduction letters were send by e-mail to enterprises (enterprise managers) with available e­mail contact, all participants were contacted by mail with guidelines for completion of web survey. In extreme situations (e.g. no internet connection available) respondents were allowed to complete survey on printed questionnaire or via CATI interview.

Before sending data was verified on the Acceptance platform of eDAMIS.

Grossing-up procedures

Grossing-up weights are calculated by performing the three-step approach, usually used in business surveys:

1) Calculation of the design weight as the reverse value of the inclusion probabilities

2) Adjustment for the non-response. The non-response adjustment factors were calculated by assuming missing at random mechanism, meaning that the factors were uniform inside the strata.

3) Weights calibration. Weights were calibrated according to the following two auxiliary variables from the Business register: Turnover and Number of employees. Calibration for Number of employees was performed on the level of NACE Activity group * Size class, while the calibration for Turnover was performed only on the level of Size class.

Calmar software was used to implement the calibration procedure.

Not applicable.

Not applicable

Not applicable

Problems encountered and lessons to be learnt: