Statistical Office of the Republic of Serbia

Information Society Statistics unit

Milana Rakica 5, Belgrade, Serbia

Data on the Information and Communication Technologies (ICT) usage and e-commerce in enterprises are survey data. They are collected by the National Statistical Institutes or Ministries and are in principle based on Eurostat's annual model questionnaires on ICT usage and e-commerce in enterprises.

Large part of the data collected is used to measure the progress in the implementation of one of the main political priorities of the European Commission for 2019 to 2024 – A Europe fit for the digital age. Part of this is the "European strategy for data", envisioning a single market for data to ensure the EU's global competitiveness and data sovereignty, in which context a comprehensive set of new rules for all digital services was proposed: the Digital Services Act and the Digital Markets Act, which are centrepieces of the EU digital strategy. Furthermore, the Commission and the High Representative of the Union for Foreign Affairs and Security Policy presented a new “EU cybersecurity strategy”, which is intended to bolster the EU's collective resilience against cyber threats, safeguard a global and open internet and protect EU values and the fundamental rights of its people. Furthermore, data will allow monitoring the progress towards  A Europe fit for the digital age, one of the six priorities for the period 2019-2024 of the von der Leyen European Commission.

The aim of the European survey on ICT usage and e-commerce in enterprises is to collect and disseminate harmonised and comparable information at European level.

 NACE Rev.2 2008

All economic activities in the scope of Annex I of the Commission Regulation are intended to be included in the general survey, covering enterprises with 10 or more employees and self-employed persons. These activities are: NACE Rev. 2 sections C, D, E, F, G, H, I, J, L, M and N, division 95.1.

For micro-enterprises see the sub-concepts below.

Micro-enterprises are not covered in the survey.

The model questionnaire on ICT usage and e-commerce in enterprises provides a large variety of variables covering among others the following areas:

-          Access to and use of the Internet

-          E-commerce and e-business

-          Use of cloud computing services

-          Artificial Intelligence

-          Other topics: Data utilisation, sharing, analytics and trading, Invoicing.

The annual model questionnaires and the European businesses statistics compliers’ manual for ICT usage and e-commerce in enterprises comprise definitions and explanations regarding the topics of the survey.

Enterprise

Target Population

As required by Annex of the Commission Implementing Regulation, enterprises with 10 or more employees and self-employed persons shall be covered by the survey.

For micro-enterprises see the sub-concepts below.

Not applicable.

All territory of the Republic of Serbia (without Kosovo)

Years 2022 and 2023.

Not applicable

Percentages of enterprises, Percentages of turnover, Percentages of employees and self-employed persons. Million RSD

We have used same reference period which was defined in the model questionnaire.

The legal basis for the 2023 EU survey on the use of ICT in enterprises is the Regulation (EU) 2019/2152 of the European Parliament and of the Council of 27 November 2019 on European business statistics establishing a common framework for European statistics relating to enterprises.

Complementary national legislation constituting the legal basis for the survey on the use of ICT in Republic of Serbia:

Law on Official Statistics (“Official Gazette of the Republic of Serbia, No 104/09) and Plan of Official Statistics (“Official Gazette of the Republic of Serbia” Nos 55/05, 71/05 – corrigendum, 101/07, 65/08, 16/11, 68/12 – CS, 72/12, 7/14 – CS, 44/14 and 30/18 – other law)

Complementary national legislation constituting the legal basis for the survey on the use of ICT in enterprises: Official statistics law

https://www.stat.gov.rs/media/2322/zakon_o_statisticie.pdf

Other authorized producers of official statistics shall be obliged to cooperate with the Office on the methodology of statistical surveys that are envisaged by annual implementation plans, and when required, they consult the Office on the methodology and databases that they determine. In case of disagreement between the authorized producers of official statistics and the Office in relation to the issues as described in paragraph 1 hereof, the Office shall be obliged to advise respectively the Government, in writing and within 30 days after the disagreement was stated. Right to access individual data of other authorized producers of official statistics.

The Office shall have the right to access individual data resulting from the surveys of other authorized producers of official statistics, in case these data are essential for conducting the activities of official statistics or for the evaluation of data quality. The right to access individual data resulting from the surveys of the National Bank of Serbia shall be granted with prior written consent by the Governor of the National Bank of Serbia. Right to access administrative data sources

The Office shall have the right to access all administrative data sources in charge of government authorities, including identification codes, if necessary, as well as the right to access the data collected by constant monitoring and observation method, unless their statistical use is explicitly prohibited by law. The holders of administrative data sources and of data collected by constant monitoring and observation method are obliged to forward the data to the Office in accordance with the adopted Plan and in a way as agreed that shall entail no extra costs for the data holders, except in cases when these data are subject to special processing.

Regulation (EC) No 223/2009 on European statistics (recital 24 and Article 20(4)) of 11 March 2009 (OJ L 87, p. 164), stipulates the need to establish common principles and guidelines ensuring the confidentiality of data used for the production of European statistics and the access to those confidential data with due account for technical developments and the requirements of users in a democratic society.

At national level : 

The data collected, processed and stored for the purpose of official statistics shall be considered as confidential when a physical person or a legal entity could be identified, directly or indirectly, by name (title), address or identification number. The authorized producers of official statistics shall use all means to preclude any possible, direct or indirect, individual identification of a reporting unit. Confidential data may be used for statistical purposes only. Therefore, the authorities of the republic government, the autonomous provinces’ government, the local government and other public authorities may not use the data and information compiled as official statistics for the purpose of defining the rights and obligations of a reporting unit.

Data are transmitted via eDamis (encrypted) and delivered to a secure environment where they are treated. Flags are added for confidentiality in case results must not be disclosed.

At national level : 

The authorized producers of official statistics are obliged to take all relevant administrative, technical and organizational measures required to protect confidential data from illegal access, disclosure or use. In a special set of regulations or other legal act, the authorized producers of official statistics shall define in more details the measures and procedures of data confidentiality protection, in accordance with this law.

Access to confidential data shall be restricted to the persons that in line of their duty produce official statistics and to the extent to which these data are required for the production of official statistics. The persons who in line of their duty have access to confidential data are under obligation to conform to the provisions of this law, even upon cessation of their term of office with statistical authorities.

There is a release calendar for the statistical outputs 

https://www.stat.gov.rs/en-US/calendar

https://www.stat.gov.rs/media/3409/dissemination-policy.pdf

Annual

Not available

https://publikacije.stat.gov.rs/G2023/Pdf/G202316018.pdf 

See detailed section 10.3.1.

Results for selected variables collected in the framework of this survey are available for all participating countries on Digital economy and society of Eurostat website.

At national level :

Not available

Not available

Not requested

Not requested

 https://data.stat.gov.rs/Metadata/27_IKT/Html/2703_ESMS_G0_2023_2.html

Not requested

1. The Methodological Manual provides guidelines and standards for the implementation of the survey in the Member States. The manual is updated and adapted every year according to the content of the model questionnaire. Furthermore, the use of the Eurostat model questionnaire improves comparability of the results of the survey among participating countries.

2. Procedure for hiring interviewers (Tender and subsequent Selection Decisions published on the Intranet)

3. Training of interviewers for ICT research

4. Continuous control of interviewers (listening, daily control of survey results)

5. Validation according to Eurostat rules

6. Preparation of a national Quality Report and reference metadata (used for quality assessment)

7. Conducting self-assessment by filling out a self-assessment questionnaire and preparing an accompanying report (self-assessment) - also for quality assessment

The European businesses statistics compliers’ manual for ICT usage and e-commerce in enterprises provides guidelines and standards for the implementation of the surveys. It is updated every year according to the changed contents of the model questionnaires.

At national level :

The SORS quality management system is relied on the Serbian official statistics mission and vision, as well as on the European Statistics Code of Practice – CoP and the Total Quality Management – TQM principles, which together make the common quality framework of the European Statistical system (ESS). Based on this framework SORS prepared main strategic documents on quality, like Declaration on quality, Quality policy, Quality strategy, as well as different guidelines and procedures for ensuring quality of statistical production.

 For more information, please see the documents at http://www.stat.gov.rs/en-US/o-nama/sistem-upravljanja-kvalitetom.

European level :

At European level, the recommended use of the annual Eurostat model questionnaire aims at improving comparability of the results among the countries that conduct the survey on ICT usage and e-commerce in enterprises. Moreover, the Methodological Manual provides guidelines and clarifications for the implementation of the surveys.

National level :

Indicators of ICT usage in enterprises are produced in accordance with methodological requirements and standards. Data are compared with previous and corresponding period and checks with different source are made.  The publication schedule is fixed and announced in advance.

European level : 

At European level, European Commission users (e.g. DG CNECT, DG GROW, DG JUST, DG REGIO, DG JRC) are the principal users of the data on ICT usage and e-commerce in enterprises and contribute in identifying/defining the topics to be covered. Hence, main users are consulted regularly (at hearings, task forces, ad hoc meetings) for their needs and are involved in the process of the development of the model questionnaires at a very early stage.

User needs are considered throughout the whole discussion process of the model questionnaires aiming at providing relevant statistical data for monitoring and benchmarking of European policies.

National level :

Main user at national level is RATEL (Republic Agency for Electronic Communications) and Ministry of Trade, Tourism and Telecommunications.

European level : 

At European level, contacts within the Commission, the OECD and other stakeholders give a clear picture about the key users' satisfaction as to the following data quality aspects: accuracy and reliability of results, timeliness, satisfactory accessibility, clarity and comparability over time and between countries, completeness and relevance. Overall users have evaluated positively (good, very good) the data quality on the ICT usage and e-commerce in enterprises.

National level :

RATEL (Republic Agency for Electronic Communications) has evaluated positively the data quality on the ICT usage in enterprises.

Detailed information is available in “ Annex I _ Completeness “ excel file - related to questionnaire, coverage, additional questions.

Not requested. 

Comments on reliability and representativeness of results and completeness of dataset

These comments reflect overall standard errors reported for the indicators and breakdowns in section 13.2.1 (Sampling error - indicators) and the rest of the breakdowns for national and European aggregates, as well as other accuracy measurements. The estimated standard error should not exceed 2pp for the overall proportions and should not exceed 5pp for the proportions related to the different subgroups of the population (for those NACE aggregates for the calculation and dissemination of national aggregates). If problems were found, these could have implications for future surveys (e.g. need to improve sampling design, to increase sample sizes, to increase the response rates).

More detailed information is available in “ Annex II. _ Accuracy “ excel file - related to European aggregates, comments on reliability and use of flag.

For calculation of the standard error see 13.2.1.1.

Standard error (for selected indicators and breakdowns)

Precision measures related to variability due to sampling, unit non-response (the size of the subset of respondents is smaller than the size of the original sample) and other (imputation for item non-response, calibration etc.) are not (yet) required from the Member states for all indicators.  Eurostat will make basic assumptions to compute these measures for all indicators produced (e.g. stratified random sampling assuming as strata the crossing of the variables “Number of employees and self-employed persons” and “Economic Activity” as it was defined in the 3 tables of section 18.1).

More detailed information is available in“ Sample and standard error tables 2023 “ excel file – worksheets starting with “Standard error".

Calculation of the standard error

Various methods can be used for the calculation of the standard error for an estimated proportion. The aim is to incorporate into the standard error the sampling variability but also variability due to unit non-response, item non-response (imputation), calibration etc. In case of census / take-all strata, the aim is to calculate the standard errors comprising the variability due to unit non-response and item non-response.

See detailed sections below.

See concept 18.1.1. A) Description of  frame population.

There was no over-coverage.

Not requested

No measurement errors detected. 

See detailed sections below.

See detailed sub-concepts below.

The following table contains the number of units (i.e. enterprises), by type of response to the survey and by the percentage of these values in relation to the gross sample size.

Not applicable.

Not available 

Not available 

Other comments relating to the item non-response

No processing errors were detected.

Not requested

See detailed section below.

The time lag will be T+0 for indicators referring to 2023 (most of variables), T+10 months for indicators referring to 2022 e.g. e-commerce.

European level : 

Data are to be delivered to Eurostat in the fourth quarter of the reference year (due date for the finalised dataset is 5th October). European results are released before the end of the survey year or in the beginning of the year following the survey year (T=reference year, T+0 for indicators referring to the current year, T+12 months for other indicators referring to the previous year e.g. e-commerce).

At national level : 

Data are released in second half of October for to survey year

 See detailed section below.

05. October

The model questionnaire is generally used by the countries that conduct the survey on ICT usage and e-commerce in enterprises. Due to (small) differences in translation, in the used survey vehicle, in non-response treatment or different routing through the questionnaire, some results for some countries may be of reduced comparability. In these cases, notes are added in the data.

Detailed information on differences in the wording of the questions in the national questionnaires is available in “ Annex I _ Completeness “ excel file - related to questionnaire, coverage, additional questions.

Not applicable

See section below.

The length of comparable time series depends on the module and the variable considered within each survey module. Additional information is available in annexes attached to the European metadata.

Not available 

Not applicable

 Not applicable

Not applicable

Not applicable

https://www.stat.gov.rs/media/2283/opsta-politika-revizije-rzs-1.pdf

Not available 

https://www.stat.gov.rs/media/2283/opsta-politika-revizije-rzs-1.pdf

 Not requested

A) Frame population description and distribution

For more information see concept 18.1.1.

B) Sampling design - Sampling method

Description of the sampling method used (e.g. stratified random sample, quota sampling, cluster sampling; one-stage or two-stage sampling) and information which variables were used to stratify, the categories of those variables, in particular for the NACE Rev. 2 categories related to the "possible calculation of European aggregates", and the final number of strata: 

In this survey, the stratified simple random sample is used. Strata are defined by cross-classifying: 

- 4 territorial units NUTS2 level (Belgrade Region, Vojvodina Region, Šumadija and Western Serbia, Southern and Eastern Serbia);

- 3 size classes as defined in balance sheets (small, medium, and large enterprises);

- 32 aggregated classes defined according to the Classification of Activities

(agg01 = 10,11,12; agg02=13,14,15; agg03=16,17,18; agg04=19; agg05=20; agg06=21, agg07=22,23; agg08=24,25; agg09=26; agg10=27; agg11=28; agg12=29,30; agg13=31,32,33; agg14=35; agg15=36,37,38,39; agg16=41,42,43; agg17=45; agg18=46; agg19=47; agg20=49,50,51,52,53; agg21=55; agg22=56; agg23=58,59,60; agg24=61; agg25=62,63; agg26=68; agg27=69,70,71; agg28=72; agg29=73,74,75; agg30=77,78,80,81,82; agg31=79; agg32=951); 

- 2 size classes according to the reported turnover in balance sheets (census units and noncensus that are randomly selected).

The final number of strata is 376.

C) Gross sample distribution

More detailed information is available in “ Sample and standard error tables 2023  “ excel file (Worksheet: GROSS SAMPLE)

D) Net sample distribution

More detailed information is available in “ Sample and standard error tables 2023  “ excel file (Worksheet: NET SAMPLE)

A) Description of frame population

 B) Frame population distribution

More detailed information is available in “ Sample and standard error tables 2023  “ excel file (Worksheet: FRAME POPULATION)

Annual

See detailed sections below.

Telephone interview

Data quality checks are done by manual revision of questionnaires and procedures of logical control. Collected data are first checked and verified during the process of data entering. After that quality checking is carried out by using a set of strict manual and electronic logical and calculation controls. Reports that fail to meet the quality standards are subject to verification and are corrected as required.

Obtained results from the actual reporting period are compared with the previous and the corresponding reporting period of the previous year. If inconsistencies are detected the data must be verified at micro level again by using raw data of the questionnaire and possibly changed.

For logical control and corrections of errors is used Integrated Survey Tool (IST). IST is a software package that includes the process of data entry, logical control and production of output tables.

Grossing-up procedures

No grossing up procedures have been performed other than standard weighting procedure.

SORS does not perform imputations

Not applicable

Not applicable

Problems encountered and lessons to be learnt: