National Statistical Institute

“Labour statistics, research and development, innovation and information society” department

2, Panayot Volov str., Sofia 1038, Bulgaria

Data on the Information and Communication Technologies (ICT) usage and e-commerce in enterprises are survey data. They are collected by the National Statistical Institutes or Ministries and are in principle based on Eurostat's annual model questionnaires on ICT usage and e-commerce in enterprises.

Large part of the data collected is used to measure the progress in the implementation of one of the main political priorities of the European Commission for 2019 to 2024 – A Europe fit for the digital age. Part of this is the "European strategy for data", envisioning a single market for data to ensure the EU's global competitiveness and data sovereignty, in which context a comprehensive set of new rules for all digital services was proposed: the Digital Services Act and the Digital Markets Act, which are centrepieces of the EU digital strategy. Furthermore, the Commission and the High Representative of the Union for Foreign Affairs and Security Policy presented a new “EU cybersecurity strategy”, which is intended to bolster the EU's collective resilience against cyber threats, safeguard a global and open internet and protect EU values and the fundamental rights of its people. Furthermore, data will allow monitoring the progress towards  A Europe fit for the digital age, one of the six priorities for the period 2019-2024 of the von der Leyen European Commission.

The aim of the European survey on ICT usage and e-commerce in enterprises is to collect and disseminate harmonised and comparable information at European level.

 NACE Rev.2 2008

All economic activities in the scope of Annex I of the Commission Regulation are intended to be included in the general survey, covering enterprises with 10 or more employees and self-employed persons. These activities are: NACE Rev. 2 sections C, D, E, F, G, H, I, J, L, M and N, division 95.1.

For micro-enterprises see the sub-concepts below.

Micro-enterprises are not included in the survey.

The model questionnaire on ICT usage and e-commerce in enterprises provides a large variety of variables covering among others the following areas:

-          Access to and use of the Internet

-          E-commerce and e-business

-          Use of cloud computing services

-          Artificial Intelligence

-          Other topics: Data utilisation, sharing, analytics and trading, Invoicing.

The annual model questionnaires and the European businesses statistics compliers’ manual for ICT usage and e-commerce in enterprises comprise definitions and explanations regarding the topics of the survey.

Enterprise

Target Population

As required by Annex of the Commission Implementing Regulation, enterprises with 10 or more employees and self-employed persons shall be covered by the survey.

For micro-enterprises see the sub-concepts below.

Not applicable.

Entire territory of the country is covered.

Data for a specific set of variables were delivered on NUTS 2 regional level.

Years 2022 and 2023.

Not applicable

Percentages of enterprises, Percentages of turnover, Percentages of employees and self-employed persons.

The reference periods in the national survey are the same as in the Model Questionnaire.

Complementary national legislation constituting the legal basis for the survey on the use of ICT in enterprises:

Annual National statistical programme adopted by the Council of ministers.

Not applicable.

Regulation (EC) No 223/2009 on European statistics (recital 24 and Article 20(4)) of 11 March 2009 (OJ L 87, p. 164), stipulates the need to establish common principles and guidelines ensuring the confidentiality of data used for the production of European statistics and the access to those confidential data with due account for technical developments and the requirements of users in a democratic society.

At national level : 

The national Statistics Act (Chapter 6 “Protection of secrecy”) guarantees the protection of statistical confidentiality and the use of individual data collected through statistical surveys for statistical purposes only.

Data are transmitted via eDamis (encrypted) and delivered to a secure environment where they are treated. Flags are added for confidentiality in case results must not be disclosed.

At national level : 

According to art. 25, p. 3 of the Statistics Act "statistical information which aggregates data about less than three statistical units or about a population in which the relative share of the value of a surveyed  parameter of a single unit exceeds 85 per cent of the total value of such parameter for all units in the population must not be disclosed".

Release Calendar presenting the outputs of the statistical surveys carried out by NSI is publicly accessible on the NSI website. 

The calendar is available on the NSI website: https://www.nsi.bg/en/content/44/basic-page/release-calendar

The data on ICT usage and e-commerce in enterprises are disseminated according to the Rules for Dissemination of Statistical Products and Services of the NSI with no deviations from the general policy. The Rules are available on the NSI website.

Annual

Regular news release for dissemination of results of the 2023 survey on ICT usage and e-commerce in enterprises is published in December on the NSI website.

National results of the survey are disseminated annually in accordance to the National Release Calendar. 

Other publications containing ICT survey data are available on the NSI website, section Information Society, ICT usage in enterprises, Publications.

See detailed section 10.3.1.

Results for selected variables collected in the framework of this survey are available for all participating countries on Digital economy and society of Eurostat website.

At national level :

Selected national data tables of the survey on ICT usage and e-commerce in enterprises are accessible via:

  - NSI website: https://www.nsi.bg/en/content/2841/ict-usage-enterprises

  - Information System INFOSTAT: https://infostat.nsi.bg/infostat/pages/module.jsf?x_2=241&lang=en 

Not applicable.

Not requested

Not requested

National reference metadata files are available on the NSI website, section Information Society, ICT usage in enterprises, Metadata and methodology: https://www.nsi.bg/en/content/2841/ict-usage-enterprises

Not requested

Quality related report of the survey on ICT usage and e-commerce in enterprises is available on the NSI website, section Information Society, ICT usage in enterprises, Quality reports: https://www.nsi.bg/en/content/2841/ict-usage-enterprises

The European businesses statistics compliers’ manual for ICT usage and e-commerce in enterprises provides guidelines and standards for the implementation of the surveys. It is updated every year according to the changed contents of the model questionnaires.

At national level :

There is a quality management system in the NSI in accordance with the requirements of ISO 9001:2015. Since 2019, an extended scope of the system has been approved, including receiving, data collection, processing, storing, analyzing, providing and disseminating of statistical information. Documents related to the quality management in the NSI are available on the NSI website: https://nsi.bg/en/content/471/basic-page/quality.

Quality of the data on ICT usage and e-commerce in enterprises is assured by strictly following the national procedures and the Methodological manual of Eurostat in the field of the information society statistics, which ensures conformity with the standard quality criteria - relevance, accuracy, reliability, timeliness, comparability and coherence. 

European level :

At European level, the recommended use of the annual Eurostat model questionnaire aims at improving comparability of the results among the countries that conduct the survey on ICT usage and e-commerce in enterprises. Moreover, the Methodological Manual provides guidelines and clarifications for the implementation of the surveys.

National level :

The quality management system certified in accordance with ISO 9001:2015, application of the uniform methodological documents, rules and procedures at all stages of the data production ensure very good quality of the data. Data for all mandatory and optional variables are provided on time to Eurostat as required by the Commission Implementing Regulation and respecting the quality criteria. 

European level : 

At European level, European Commission users (e.g. DG CNECT, DG GROW, DG JUST, DG REGIO, DG JRC) are the principal users of the data on ICT usage and e-commerce in enterprises and contribute in identifying/defining the topics to be covered. Hence, main users are consulted regularly (at hearings, task forces, ad hoc meetings) for their needs and are involved in the process of the development of the model questionnaires at a very early stage.

User needs are considered throughout the whole discussion process of the model questionnaires aiming at providing relevant statistical data for monitoring and benchmarking of European policies.

National level :

User needs at national level are considered through a consultation of the draft National statistical programme with the main groups of users (ministries, local authorities, scientific community, etc.). 

European level : 

At European level, contacts within the Commission, the OECD and other stakeholders give a clear picture about the key users' satisfaction as to the following data quality aspects: accuracy and reliability of results, timeliness, satisfactory accessibility, clarity and comparability over time and between countries, completeness and relevance. Overall users have evaluated positively (good, very good) the data quality on the ICT usage and e-commerce in enterprises.

National level :

At national level, a general survey on user satisfaction is conducted annually. Information is available on the NSI website, section Products and services, User satisfaction: https://www.nsi.bg/en/content/16537/questionnaire-survey-user-satisfaction-offered-statistical-data-products-and-services

Dedicated user satisfaction survey for the ICT domain is not conducted.

Detailed information is available in “ Annex I _ Completeness “ excel file - related to questionnaire, coverage, additional questions.

Not requested. 

Comments on reliability and representativeness of results and completeness of dataset

These comments reflect overall standard errors reported for the indicators and breakdowns in section 13.2.1 (Sampling error - indicators) and the rest of the breakdowns for national and European aggregates, as well as other accuracy measurements. The estimated standard error should not exceed 2pp for the overall proportions and should not exceed 5pp for the proportions related to the different subgroups of the population (for those NACE aggregates for the calculation and dissemination of national aggregates). If problems were found, these could have implications for future surveys (e.g. need to improve sampling design, to increase sample sizes, to increase the response rates).

More detailed information is available in “ Annex II. _ Accuracy “ excel file - related to European aggregates, comments on reliability and use of flag.

For calculation of the standard error see 13.2.1.1.

Standard error (for selected indicators and breakdowns)

Precision measures related to variability due to sampling, unit non-response (the size of the subset of respondents is smaller than the size of the original sample) and other (imputation for item non-response, calibration etc.) are not (yet) required from the Member states for all indicators.  Eurostat will make basic assumptions to compute these measures for all indicators produced (e.g. stratified random sampling assuming as strata the crossing of the variables “Number of employees and self-employed persons” and “Economic Activity” as it was defined in the 3 tables of section 18.1).

More detailed information is available in“ Sample and standard error tables 2023 “ excel file – worksheets starting with “Standard error".

Calculation of the standard error

Various methods can be used for the calculation of the standard error for an estimated proportion. The aim is to incorporate into the standard error the sampling variability but also variability due to unit non-response, item non-response (imputation), calibration etc. In case of census / take-all strata, the aim is to calculate the standard errors comprising the variability due to unit non-response and item non-response.

See detailed sections below.

See concept 18.1.1. A) Description of  frame population.

Over-coverage rate is 6.1%.

Not requested

Following measures are taken for prevention of measurement errors:

 -  Questionnaire design follows the 2023 model questionnaire provided by Eurostat, in order to assure the comparability of the data. The questions are duly translated and adapted to the national specifics. The questionnaire is tested internally prior to data collection;

 - Data entry program is designed to guarantee correctness and consistency of the initial data (logical controls, acceptable value, etc.);

 - Interviewers engaged for the survey are experienced staff of the regional offices of NSI. They are instructed and assisted by the regional coordinators of the survey prior to and during the fieldwork.

See detailed sections below.

See detailed sub-concepts below.

The following table contains the number of units (i.e. enterprises), by type of response to the survey and by the percentage of these values in relation to the gross sample size.

In order to reduce the unit non-response the following measures are taken:

 - advance notification prior to the survey and following reminders are sent to the respondents by e-mail;

 - phone assistance by statistical experts for methodological and technical issues is provided;

 - if the enterprise is not able or not willing to fill in the questionnaire online, it is contacted by phone by an interviewer;

 - the interviewers are experienced staff of the NSI's regional offices, which increase the confidence of the respondents.

Not applicable.

There are controls to each question in the web questionnaire and the respondent is not allowed to move to the next question if one or more of the answer options are not completed. For that reason there is minor or no item non-response.

Other comments relating to the item non-response

Processing errors are minimized by the following measures: 

 -  during data collection - interviewers check the correctness of the information filled-in and re-contact the respondents, if necessary;

 -  during data entry - consistency and logical controls are implemented in the data entry software and incorrect data cannot be entered;

 -  during data processing - additional checks for logical consistence and year-to-year checks are carried out over the aggregated data.

Not requested

See detailed section below.

Not applicable

European level : 

Data are to be delivered to Eurostat in the fourth quarter of the reference year (due date for the finalised dataset is 5th October). European results are released before the end of the survey year or in the beginning of the year following the survey year (T=reference year, T+0 for indicators referring to the current year, T+12 months for other indicators referring to the previous year e.g. e-commerce).

At national level : 

National aggregated data are delivered to Eurostat without a delay.

 See detailed section below.

National data are published according to the target date set in the Release calendar.

The model questionnaire is generally used by the countries that conduct the survey on ICT usage and e-commerce in enterprises. Due to (small) differences in translation, in the used survey vehicle, in non-response treatment or different routing through the questionnaire, some results for some countries may be of reduced comparability. In these cases, notes are added in the data.

Detailed information on differences in the wording of the questions in the national questionnaires is available in “ Annex I _ Completeness “ excel file - related to questionnaire, coverage, additional questions.

Data for specific set of variables were delivered on NUTS 2 regional level. There is no problem of comparability across the country’s regions.

Not applicable

See section below.

The length of comparable time series depends on the module and the variable considered within each survey module. Additional information is available in annexes attached to the European metadata.

No changes that may have an impact on the comparability over time are made in the national questionnaire.

Not applicable

 Not applicable

Not applicable

Not applicable

There is no general data revision policy.

The published data from the survey on ICT usage in enterprises are considered final and are not subject to revision. However, the data could be revised under unforeseen circumstances.

 Not requested

A) Frame population description and distribution

For more information see concept 18.1.1.

B) Sampling design - Sampling method

Description of the sampling method used (e.g. stratified random sample, quota sampling, cluster sampling; one-stage or two-stage sampling) and information which variables were used to stratify, the categories of those variables, in particular for the NACE Rev. 2 categories related to the "possible calculation of European aggregates", and the final number of strata: 

The sampling method used is one-stage stratified random sampling. The stratification is done by NUTS 2 (6 categories), economic activity (33 categories, incl. NACE categories related to European aggregates and class 47.91), ICT sector (2 categories) and number of persons employed broken down into three size classes (10-49; 50-249; 250+ persons employed). The total number of non-empty strata is 510. The number of enterprises per stratum is determined using Neyman allocation method. The enterprises in the size group with 250+ persons employed are surveyed exhaustively. Minimum sample size per stratum is 3 enterprises. Module Complex Samples in SPSS is used for sample selection. The sample is independent and there are no procedures used for coordination or non-overlapping with samples of other surveys, however the previous year’s sample of small and medium enterprises is taken into account, i.e. rotation is applied in case the number of enterprises in stratum allows that.

C) Gross sample distribution

More detailed information is available in “ Sample and standard error tables 2023  “ excel file (Worksheet: GROSS SAMPLE)

D) Net sample distribution

More detailed information is available in “ Sample and standard error tables 2023  “ excel file (Worksheet: NET SAMPLE)

A) Description of frame population

 B) Frame population distribution

More detailed information is available in “ Sample and standard error tables 2023  “ excel file (Worksheet: FRAME POPULATION)

Annual

See detailed sections below.

Server based EDIT validation tool is used to validate the data. Additional verifications via year-to-year and consistency checks are made on the aggregated data.

Grossing-up procedures

All enterprises, both in the frame population and in the net sample, are post-classified into one of the strata, defined by crossing of NUTS2, NACE groups for national and European aggregates and size classes (according to the number of employees). Final weight of each enterprise is calculated as a ratio between the number of enterprises in its stratum in the frame population and the number of enterprises in the same stratum in the net sample (responding enterprises). Enterprises with more than 250 persons employed are surveyed exhaustively, nevertheless final weights are calculated for them as well, based on the number of responding enterprises. Preliminary SBS data for year 2022 as of 16.08.2023 is used as a frame population for grossing-up in order to present more current structure of the enterprises population. 

Following types of weights are calculated to gross up the data depending on the variable type:

1. weight based on the number of enterprises;

2. weight based on the persons employed - for grossing-up the variables related to persons employed;

3. weight based on the total turnover - for grossing-up the currency variables.

Imputation was not used.

Not applicable

Not applicable

Problems encountered and lessons to be learnt: