ICT usage in enterprises (isoc_e)

National Reference Metadata in SIMS structure for INFOSOC Enterprises

Compiling agency: Statistics Norway


Eurostat metadata
Reference metadata
1. Contact
2. Metadata update
3. Statistical presentation
4. Unit of measure
5. Reference Period
6. Institutional Mandate
7. Confidentiality
8. Release policy
9. Frequency of dissemination
10. Accessibility and clarity
11. Quality management
12. Relevance
13. Accuracy
14. Timeliness and punctuality
15. Coherence and comparability
16. Cost and Burden
17. Data revision
18. Statistical processing
19. Comment
Related Metadata
Annexes (including footnotes)



For any question on data and metadata, please contact: Eurostat user support

Download


1. Contact Top
1.1. Contact organisation

Statistics Norway

1.2. Contact organisation unit

Division for business dynamics statistics

1.5. Contact mail address

Statistisk sentralbyrå
Postboks 2633 St. Hanshaugen
0131 Oslo

Norway


2. Metadata update Top
2.1. Metadata last certified 21/03/2024
2.2. Metadata last posted 21/03/2024
2.3. Metadata last update 21/03/2024


3. Statistical presentation Top
3.1. Data description

Data on the Information and Communication Technologies (ICT) usage and e-commerce in enterprises are survey data. They are collected by the National Statistical Institutes or Ministries and are in principle based on Eurostat's annual model questionnaires on ICT usage and e-commerce in enterprises.

Large part of the data collected is used to measure the progress in the implementation of one of the main political priorities of the European Commission for 2019 to 2024 – A Europe fit for the digital age. Part of this is the "European strategy for data", envisioning a single market for data to ensure the EU's global competitiveness and data sovereignty, in which context a comprehensive set of new rules for all digital services was proposed: the Digital Services Act and the Digital Markets Act, which are centrepieces of the EU digital strategy. Furthermore, the Commission and the High Representative of the Union for Foreign Affairs and Security Policy presented a new “EU cybersecurity strategy”, which is intended to bolster the EU's collective resilience against cyber threats, safeguard a global and open internet and protect EU values and the fundamental rights of its people. Furthermore, data will allow monitoring the progress towards  A Europe fit for the digital age, one of the six priorities for the period 2019-2024 of the von der Leyen European Commission.

The aim of the European survey on ICT usage and e-commerce in enterprises is to collect and disseminate harmonised and comparable information at European level.

 

Name of data collection

Data collection on ICT usage and e-commerce in enterprises 2023

Name in Norwegian: Bruk av IKT i næringslivet 2023

Click on the hyperlink below for more information about the statistics

ICT usage in enterprises – SSB

3.2. Classification system

 NACE Rev.2 2008 

3.3. Coverage - sector

All economic activities in the scope of Annex I of the Commission Regulation are intended to be included in the general survey, covering enterprises with 10 or more employees and self-employed persons. These activities are: NACE Rev. 2 sections C, D, E, F, G, H, I, J, L, M and N, division 95.1.

For micro-enterprises see the sub-concepts below.

3.3.1. Coverage-sector economic activity for micro-enterprises - All NACE Rev. 2 categories are covered
3.3.2. Coverage sector economic activity for micro-enterprises - If not all activities were covered, which ones were covered?

Micro-enterprises is not included in the survey.

3.4. Statistical concepts and definitions

The model questionnaire on ICT usage and e-commerce in enterprises provides a large variety of variables covering among others the following areas:

-          Access to and use of the Internet

-          E-commerce and e-business

-          Use of cloud computing services

-          Artificial Intelligence

-          Other topics: Data utilisation, sharing, analytics and trading, Invoicing.

The annual model questionnaires and the European businesses statistics compliers’ manual for ICT usage and e-commerce in enterprises comprise definitions and explanations regarding the topics of the survey.

3.5. Statistical unit

Enterprises

3.6. Statistical population

Target Population

As required by Annex of the Commission Implementing Regulation, enterprises with 10 or more employees and self-employed persons shall be covered by the survey.

For micro-enterprises see the sub-concepts below.

3.6.1. Coverage of micro-enterprises
No
3.6.2. Breakdown between size classes [0 to 1] and [2 to 9]
No
3.6.3. If for micro-enterprises different size delimitation was used, please indicate it.

Not applicable

3.7. Reference area

All the territory of the mainland country was included in the population. In other words, the survey covered all the counties in Norway. However, the islands Svalbard, Bjørnøya and Jan Mayen were not included. Neither was the Continental shelf included. The percentage of the target population not covered was 0.05.

Data for a specific set of variables were delivered on NUTS 2 regional level.

3.8. Coverage - Time

Years 2022 and 2023

3.9. Base period

Not applicable


4. Unit of measure Top

Percentages of enterprises, Percentages of turnover, Percentages of employees and self-employed persons. Million euro.


5. Reference Period Top

The reference periods defined in the 2023 model questionnaire for the variables were followed in the national survey.


6. Institutional Mandate Top
6.1. Institutional Mandate - legal acts and other agreements

Complementary national legislation constituting the legal basis for the survey on the use of ICT in enterprises:

THE EUROPEAN COMMISSION,
Having regard to the Treaty on the Functioning of the European Union,
Having regard to Regulation (EU) 2019/2152 of the European Parliament and of the Council of 27 November 2019 on European business statistics, repealing 10 legal acts in the field of business statistics (OJ L 327, 17.12.2019, p. 1.), and in particular Article 7(1) and Article 17(6) thereof,
Whereas:
(1) In order to produce pursuant to Annex II to Regulation (EU) 2019/2152 data on the topic ICT usage and e-commerce, listed in Annex I to that Regulation, based on comparable and harmonised data, and to ensure the correct implementation of the topic ICT usage and e-commerce by the Member States, the Commission is to specify the variables, measurement units, statistical population, classifications and breakdowns, as well as data transmission deadlines for that data.
(2) Pursuant to point (a) of Article 17(4) of Regulation (EU) 2019/2152 Member States are required to provide quality and metadata reports for data transmitted under that Regulation. It is therefore necessary to establish the deadlines for submission of those reports.
(3) The measures provided for in this Regulation are in accordance with the opinion of the European Statistical System Committee,
HAS ADOPTED THIS REGULATION:
Article 1
For the topic ICT usage and e-commerce, as referred to in Annex I to Regulation (EU) 2019/2152, Member States shall transmit to the Commission (Eurostat) the data for reference year 2023 in accordance with the Annex to this Regulation.
Article 2
1. The annual metadata report for reference year 2023 for the topic ICT usage and e-commerce shall be transmitted to the Commission (Eurostat) by 31 May 2023.
2. The annual quality report for the topic ICT usage and e-commerce for reference year 2023 shall be transmitted to the Commission (Eurostat) by 5 November 2023.
Article 3
This Regulation shall enter into force on the twentieth day following that of its publication in the Official Journal of the European Union.
This Regulation shall be binding in its entirety and directly applicable in all Member States.

Complementary national legislation constituting the legal basis for the survey on the use of ICT in enterprises:

The Norwegian Statistics Act (LOV-2019-06-21-32) § 10, 20

6.2. Institutional Mandate - data sharing

The NSI usually receive request from UN Conference on Trade and Development (UNCTAD) and OECD to provide some ICT-indicators in the previous years. The requested data are in accordance with the core list of ICT indicators composed by OECD and proposed by the Partnership on Measuring ICT for Development and endorsed by the United Nations Statistical Commission. We expect and intend to continue sharing data with OECD and UNCTAD.


7. Confidentiality Top
7.1. Confidentiality - policy

Regulation (EC) No 223/2009 on European statistics (recital 24 and Article 20(4)) of 11 March 2009 (OJ L 87, p. 164), stipulates the need to establish common principles and guidelines ensuring the confidentiality of data used for the production of European statistics and the access to those confidential data with due account for technical developments and the requirements of users in a democratic society.

At national level : 

The general rule is that no information shall be published in such a way that it may be traced back to identifiable units. Exceptions can be made but under no circumstances shall unit information be published to the unreasonable detriment of the unit.

In addition to European legislation and international Code of Practice, The Statistics Act § 7. The Statistics Act § 7. Statistical confidentiality in the dissemination of official statistics requires that Statistics Norway does not publish statistics in a manner that allows statistical information to be traced back to individuals or other types of statistical units.

Based on these acts and guidelines Statistics Norway have prepared rules and guidelines for treatment of data (described in 7.2.) to prevent unauthorised disclosure of data that identify an economic entity or a statistical unit either directly or indirectly.

7.2. Confidentiality - data treatment

Data are transmitted via eDamis (encrypted) and delivered to a secure environment where they are treated. Flags are added for confidentiality in case results must not be disclosed.

At national level : 

Confidentiality can be ensured by using coarser categories, by suppression (hiding values) or by perturbation (changing values). Suppression means that some values are not published and are instead replaced by a colon. In addition to the values one wishes to protect via suppression, several other values in the table must be so-called secondary suppressed to prevent the possibility of recalculation from other published values (aggregates). When applying perturbation, the published value may deviate from the real value. Several methods can be used, including rounding and noise addition.

The general rule is that information about smaller groups than those containing three units or groups completely dominated by one or two units will not be published. This has not been considered a problem for the national publishing.

In case if any breakdowns that consist of less than three units or consist of one dominant unit, all of the aggregates will be flagged with C in the CONF_STATUS in the finalized dataset, before the data transmission to Eurostat. Moreover, in the annex of the the selected indicators for calculation of standard error, all of the variables that are related to the breakdowns will be flagged with C as well. The breakdowns will not be released due to the confidential issue. 


8. Release policy Top
8.1. Release calendar

There is a release calendar for national publishing. The release date for the ICT ENT 2023, 22/09/2023, was updated in first quarter 2023 and it is publicly accessible.

8.2. Release calendar access

Hyperlink to the NSI calendar for all publication: Upcoming releases and publications – SSB

The release date in the website to the ICT ENT: ICT usage in enterprises – SSB

8.3. Release policy - user access

The general data release policy of the organisation is that no external users have access to the statistics and analyses before they are published and accessible simultaneously for all users on ssb.no at 8 am. Prior to this, a minimum of three months' advance notice is given in the NSI statistics release calendar. 

The main principles of Statistics Norway’s communication and dissemination are transparency, accessibility, comprehensibility, independence and confidentiality.

Please click on the link below for more information about the main principles:

Prinsipper for kommunikasjon og formidling (ssb.no)


9. Frequency of dissemination Top

Annual


10. Accessibility and clarity Top
10.1. Dissemination format - News release

National dissemination of the news article, released 22th September 2023 by Statistics Norway:

Ett av ti foretak utfører dataanalyse selv (ssb.no)

10.2. Dissemination format - Publications

National dissemination of results, released 22th September 2023 by Statistics Norway:

Bruk av IKT i næringslivet (ssb.no)

10.3. Dissemination format - online database

See detailed section 10.3.1.

10.3.1. Data tables - consultations

Results for selected variables collected in the framework of this survey are available for all participating countries on Digital economy and society of Eurostat website.

At national level :

National data tables/databases: ICT usage in enterprises

ICT usage in enterprises. Statbank Norway (ssb.no)

 

10.4. Dissemination format - microdata access

Description of national rules and procedures for microdata access:

Access to data from Statistics Norway (ssb.no)

10.5. Dissemination format - other

Not requested

10.5.1. Metadata - consultations

Not requested

10.6. Documentation on methodology

The main methodological documents used for the survey is the Methodological Manual provided by Eurostat.

For the description of national statistics regarding metadata, methodology, summary, etc., please click on the attached link (see the details in "About the statistics"):

ICT usage in enterprises (ssb.no)

10.6.1. Metadata completeness - rate

Not requested

10.7. Quality management - documentation

Please see the description of quality management in the 11.2 Quality management - assessment


11. Quality management Top
11.1. Quality assurance

The European businesses statistics compliers’ manual for ICT usage and e-commerce in enterprises provides guidelines and standards for the implementation of the surveys. It is updated every year according to the changed contents of the model questionnaires.

At national level :

According to the letter of allocation, Statistics Norway shall oversee the monitoring of compliance with the requirements for quality in official statistics and establish a system for following this up. The statistics programme identifies 12 producers of statistics, all of which are involved in the quality assurance work. In consultation with the Committee for Official Statistics, Statistics Norway shall establish a quality system in the first half of 2021 to ensure that work on quality assurance can start in the second half of the year. An initial report on the quality of official statistics shall be submitted to the Ministry in the first half of 2022.

This memo describes a proposal for a quality system for official statistics. The system consists of four general elements:

  1. A framework for the quality of official statistics consisting of the Statistics Act and the European Statistics Code of Practice. This is described in Section 3.
  2. Methods and tools for measuring the quality of official statistics. Section 4 gives an assessment of established methods and tools, how these can be incorporated into a quality system, and what new methods need to be developed and applied.
  3. Systems and processes for following up quality. Section 5 describes how the different elements of the system will interact and how quality can be assessed and followed up consistently.
  4. Actors and their roles. Section 6 examines the responsibilities and roles of the different actors in the quality system for official statistics.

For more information, please click on the link to the documentation below:

System for quality assurance of official statistics – SSB

11.2. Quality management - assessment

European level :

At European level, the recommended use of the annual Eurostat model questionnaire aims at improving comparability of the results among the countries that conduct the survey on ICT usage and e-commerce in enterprises. Moreover, the Methodological Manual provides guidelines and clarifications for the implementation of the surveys.

National level :

Official statistics of Statistics Norway are produced using a number of statistical methods. Statistics Norway has a department that supports the use of these and in some areas also further develops and researches methods.

  • Questionnaire methodology and user testing: Questionnaire methodology includes a number of different methods that are used iteratively. The field of study is based on a number of other academic disciplines such as social anthropology, psychology, linguistics and philosophy. The cognitive perspective on the response process is central to our work, and can be divided into four iterative phases: interpretation of meaning, retrieval of information required, assessment of information, and the selection of an answer. Our methods for user testing include: expert evaluation, focus groups, cognitive mapping, eye tracking and usability testing.
  • Sampling design: A sample survey is conducted when it is too extensive to ask the entire population. Sampling design is the planning of which units are to be included in such a survey. This includes analysing the population that is the basis for the sample, calculating sample sizes and determining the sampling method.

  • Data integration: Data integration is to combine data from different data sources, with the aim of producing new data sets that are the basis for statistics or research.
  • Data editing: Data editing is the control, scrutiny and correction of data. It includes editing of population, editing of obvious and systematic errors, selection of values ​​with large deviations and high influence and control of aggregates to be published. The methods used for data editing range from logical control of valid value range to imputation of values ​​with machine learning.

  • Estimation and weighting: Estimation is to find value for a population figure based on the information we have collected from the (sample) survey. We are usually interested in several figures - totals, averages, proportions and variances are most common - for several variables. Estimation often means that each unit in the sample is assigned a weight, this is almost always done for personal and household surveys. We can also base the estimation on a statistical model, which is common in business surveys.

  • Seasonal adjustment: Seasonal adjustment is to use statistical methods to remove systematic seasonal variations from a monthly or quarterly time series, so that the time series to the greatest possible extent expresses the real development over time. In addition, they try to remove the calendar effects that vary from year to year, such as Easter. When the data has been corrected for the seasonal conditions, one will be left with a clearer picture of the underlying development in the time series which consists of trend / cycle and irregular component.
  • Confidentiality: The Statistics Act § 7. Statistical confidentiality in the dissemination of official statistics requires that Statistics Norway does not publish statistics in a manner that allows statistical information to be traced back to individuals or other types of statistical units. Confidentiality can be ensured by using coarser categories, by suppression (hiding values) or by perturbation (changing values). Examples of coarser categories are counties instead of municipalities and age groups of five years instead of single years. Suppression means that some values are not published and are instead replaced by a colon. In addition to the values one wishes to protect via suppression, several other values in the table must be so-called secondary suppressed to prevent the possibility of recalculation from other published values (aggregates). When applying perturbation, the published value may deviate from the real value. Several methods can be used, including rounding and noise addition.

 

For more comprehensive information about overall assessment of the national methodology for quality management, please click on the links below:

Quality work in Statistics Norway: Quality work in Statistics Norway (ssb.no) 

Quality in official statistics: Quality in official statistics – SSB

International frameworks for quality in official statistics: International frameworks for quality in official statistics – SSB

 


12. Relevance Top
12.1. Relevance - User Needs

European level : 

At European level, European Commission users (e.g. DG CNECT, DG GROW, DG JUST, DG REGIO, DG JRC) are the principal users of the data on ICT usage and e-commerce in enterprises and contribute in identifying/defining the topics to be covered. Hence, main users are consulted regularly (at hearings, task forces, ad hoc meetings) for their needs and are involved in the process of the development of the model questionnaires at a very early stage.

User needs are considered throughout the whole discussion process of the model questionnaires aiming at providing relevant statistical data for monitoring and benchmarking of European policies.

National level :

Eurostat is an important user of the statistics. Ministries, Innovation Norway, Norwegian Digitalisation Agency, business organisations, research institutes, the media and international organisations such as OECD, UNCTAD are also main users of the statistics.

 

12.2. Relevance - User Satisfaction

European level : 

At European level, contacts within the Commission, the OECD and other stakeholders give a clear picture about the key users' satisfaction as to the following data quality aspects: accuracy and reliability of results, timeliness, satisfactory accessibility, clarity and comparability over time and between countries, completeness and relevance. Overall users have evaluated positively (good, very good) the data quality on the ICT usage and e-commerce in enterprises.

National level :

No information available.

12.3. Completeness

Detailed information is available in “ Annex I _ Completeness “ excel file - related to questionnaire, coverage, additional questions.

All mandatory and optional questions were included in the national questionnaire. No deviation from question / item in model questionnaire. Micro-enterprises is not included in the national survey. Turnover is collected from the last version of SBS. Employment and economic activity are collected from administrative register data.

Additional questions included in the national survey: "Use of government authorities’ data" (6 questions).
The additional questions included on behalf of the Norwegian Digitalisation Agency's needs. Please see the questions in the annex "National ICT ENT Survey 2023 - NO"



Annexes:
Annex I_Completeness 2023 - NO
12.3.1. Data completeness - rate

Not requested. 


13. Accuracy Top
13.1. Accuracy - overall

Comments on reliability and representativeness of results and completeness of dataset

These comments reflect overall standard errors reported for the indicators and breakdowns in section 13.2.1 (Sampling error - indicators) and the rest of the breakdowns for national and European aggregates, as well as other accuracy measurements. The estimated standard error should not exceed 2pp for the overall proportions and should not exceed 5pp for the proportions related to the different subgroups of the population (for those NACE aggregates for the calculation and dissemination of national aggregates). If problems were found, these could have implications for future surveys (e.g. need to improve sampling design, to increase sample sizes, to increase the response rates).

More detailed information is available in “ Annex II. _ Accuracy “ excel file - related to European aggregates, comments on reliability and use of flag.

Due to the implementation of new NACE breakdowns, several industries (that belonged to the same breakdown-group earlier) have been subdivided into own stand-along-groups. This leads to some of the  breakdowns/groups containing relatively few units, e.g. NACE C13T15 has few units in the population which led to some of the indicators are having a standard error more than 5%. Despite this, the NSI consider the provided data and results are accurate, reliable and representative.

None of selected indicators have standard error that is exceed 2pp for the overall proportions. However, some of the selected indicators have standard error that were exceeded 5pp for some small subgroups, but there is no one particular breakdown/group that consistently has a high pp of standard error that is exceeded 5pp on every indicator. The issue with high standard error for some small subgroups is often related to there were few units in the population. We have had the same issue with high standard error for some small groups earlier also - before the implantation of the new NACE breakdowns. However, each of these subgroups with a high pp of standard error representing less that 2 percent of the entire population. In 2023 there was zero unit in the population to the C19. Therefore the C19 breakdown is missing.



Annexes:
Annex II._Accuracy_2023_NO
13.2. Sampling error

For calculation of the standard error see 13.2.1.1.

13.2.1. Sampling error - indicators

Standard error (for selected indicators and breakdowns)

Precision measures related to variability due to sampling, unit non-response (the size of the subset of respondents is smaller than the size of the original sample) and other (imputation for item non-response, calibration etc.) are not (yet) required from the Member states for all indicators.  Eurostat will make basic assumptions to compute these measures for all indicators produced (e.g. stratified random sampling assuming as strata the crossing of the variables “Number of employees and self-employed persons” and “Economic Activity” as it was defined in the 3 tables of section 18.1).

More detailed information is available in“ Sample and standard error tables 2023 “ excel file – worksheets starting with “Standard error".



Annexes:
Sample and standard error tables 2023 - NO
13.2.1.1. Sampling error indicator calculation

Calculation of the standard error

Various methods can be used for the calculation of the standard error for an estimated proportion. The aim is to incorporate into the standard error the sampling variability but also variability due to unit non-response, item non-response (imputation), calibration etc. In case of census / take-all strata, the aim is to calculate the standard errors comprising the variability due to unit non-response and item non-response.

a) Name and brief description of the applied estimation approach
 Ratio model is intended to be used

 

b) Basic formula
   yhi = ßhxhi + εhi , i = 1, 2, ..., Nh and var(εhi) = xhiσ2h

 

c) Main reference in the literature
  Valliant, R., Dorfman, A.H. and Rouall, R.M. (2001). Finite Population Sampling and Inference: A Prediction approach. Wiley, New York.

 

d) How has the stratification been taken into account? 
 Please see the detailed descriptions in 18.1 B) and 18.5

 

e) Which strata have been considered? 
  Please see the detailed description in 18.5
13.3. Non-sampling error

See detailed sections below.

13.3.1. Coverage error

See concept 18.1.1. A) Description of  frame population.

No additional known shortcomings (please see 18.1 and 18.1.1).

13.3.1.1. Over-coverage - rate

No issue  concering over-coverage

13.3.1.2. Common units - proportion

Not requested

13.3.2. Measurement error

No issue concerning measurement error

13.3.3. Non response error

See detailed sections below.

13.3.3.1. Unit non-response - rate

See detailed sub-concepts below.

13.3.3.1.1. Unit response

The following table contains the number of units (i.e. enterprises), by type of response to the survey and by the percentage of these values in relation to the gross sample size.

 

Type of response Enterprises
0-9 employees and self-employed persons 10 or more employees and self-employed persons
Number % Number %
Gross sample size (as in section 3.1 C)   100%  5617 100%
1. Response (questionnaires returned by the enterprise)      5565  99.07% 
1.1 Used for tabulation and grossing up (Net sample or Final Sample; as in section 3.1 D)      5560 99.91% 
1.2 Not used for tabulation      5   0.09%
1.2.1 Out of scope (deaths, misclassified originally in the target population, etc.)      0  0%
1.2.2 Other reasons (e.g. unusable questionnaire)      5 0.09% 
2. Non-response (e.g. non returned mail, returned mail by post office)      52 0.93% 

 

Comments on unit response, if unit response is below 60%
 
13.3.3.1.2. Methods used for minimizing unit non-response

Accompanying with the web-survey, an electronic notification was sent to the respondents with information about the survey, including deadline, and legally consequences of not responding to the survey. When the deadline is reached, the respondents who have not yet submitted the survey will get a warning, and a new deadline. When the new deadline expires, and the submission is still missing, then a compulsory fine will be imposed. The fine will continue to accrue until the survey is submitted or the maximum limit for enforcement fines is reached. Throughout the data collection, the ICT ENT staff was available for any questions the respondents may have regarding the survey. In some case, the statistic responsible will send extra reminders to the important units. Unit non-response never has been a problem. Over 95 per cent of the enterprises responded in the previous years, and 99 percent response-rate in 2023.

13.3.3.1.3. Methods used for unit non-response treatment
1. No treatment for unit non-response                                           X
2. Treatment by re-weighting
2.1 Re-weighting by the sampling design strata considering that non-response is ignorable inside each stratum (the naïve model)  
2.2 Re-weighting by identified response homogeneity groups (created using sample-level information)  
2.3 Re-weighting through calibration/post-stratification (performed using population information) by the groups used for calibration/post-stratification  
3. Treatment by imputation (done distinctly for each variable/item)  
4. Method(s) and the model(s) corresponding to the above or other method(s) used for the treatment of unit non-response. (e.g. Re-weighting using Horvitz-Thompson estimator, ratio estimator or regression estimator, auxiliary variables )
 
13.3.3.1.4. Assessment of unit non-response bias

More than 99% response rate. No need for assessment.

13.3.3.2. Item non-response - rate

None (Due to that most of the items in the national survey are mandatory. If any item is missing, the survey cannot be submitted, i.e. the mandatory obligation to provide information is not fulfilled for the respondents.)

13.3.3.2.1. Methods used for item non-response treatment
1. No treatment for item non-response  X
2. Deductive imputation
An exact value can be derived as a known function of other characteristics.
 
3. Deterministic imputation (e.g. mean/median, mean/median by class, ratio-based, regression-based, single donor nearest-neighbour)
Deterministic imputation leads to estimators with no random component, that is, if the imputation were to be re-conducted, the outcome would be the same
 
4. Random imputation (e.g. hot-deck, cold-deck)
Random imputation leads to estimators with a random component, that is, if the imputation were re-conducted, it would have led to a different result
 
5. Re-weighting  
6. Multiple imputation
In multiple imputation each missing value is replaced (instead of a single value) with a set of plausible values that represent the uncertainty of the right value to impute. Multiple imputation methods offer the possibility of deriving variance estimators by taking imputation into account. The incorporation of imputation into the variance can be easily derived based on variability of estimates among the multiply imputed data sets.
 
7. Method(s) and the model(s) corresponding to the above or other method(s) used for the treatment of item non-response.
 
13.3.3.2.2. Questions or items with item response rates below 90% and other comments

Other comments relating to the item non-response

Additional issues concerning "non-response" calculation (e.g. method used in national publications).
 None

 

Questions and items with low response rates (cut-off value is 90% ) and item non-response rate.
 None
13.3.4. Processing error

None

13.3.5. Model assumption error

Not requested


14. Timeliness and punctuality Top
14.1. Timeliness

See detailed section below.

14.1.1. Time lag - first result

Not applicable

14.1.2. Time lag - final result

European level : 

Data are to be delivered to Eurostat in the fourth quarter of the reference year (due date for the finalised dataset is 5th October). European results are released before the end of the survey year or in the beginning of the year following the survey year (T=reference year, T+0 for indicators referring to the current year, T+12 months for other indicators referring to the previous year e.g. e-commerce).

At national level : 

The data was transmitted to Eurostat (via the EDAMIS4) at 3th October 2023. There was no error in the validation report. The Y2Y check file received from Eurostat afterward has been checked, and all of the changes were confirmed as reliable results. The confirmation was sent to Eurostat 19th October. Which means that the data transmitted is the final results.

 

14.2. Punctuality

 See detailed section below.

14.2.1. Punctuality - delivery and publication

Date of release of final national data: 22th September 2023

Punctuality - delivery and publication: The finalized data was delivered to Eurostat at 3th October 2023, two day before the deadline, i.e. -2 day time lag between the data delivery and the deadline. 


15. Coherence and comparability Top
15.1. Comparability - geographical

The model questionnaire is generally used by the countries that conduct the survey on ICT usage and e-commerce in enterprises. Due to (small) differences in translation, in the used survey vehicle, in non-response treatment or different routing through the questionnaire, some results for some countries may be of reduced comparability. In these cases, notes are added in the data. 

Data for specific set of variables were delivered on NUTS 2 regional level. There is no problem of comparability across the country’s regions.

Detailed information on differences in the wording of the questions in the national questionnaires is available in “ Annex I _ Completeness “ excel file - related to questionnaire, coverage, additional questions.

 No deviation from the model questionnaire that would affect comparability.

15.1.1. Asymmetry for mirror flow statistics - coefficient

Not applicable

15.2. Comparability - over time

See section below.

15.2.1. Length of comparable time series

The length of comparable time series depends on the module and the variable considered within each survey module. Additional information is available in annexes attached to the European metadata.

The changes for the fixed modules/variables from previous years are considered small and should not impact the comparability over time of results delivered to Eurostat.

15.3. Coherence - cross domain

Not applicable

15.3.1. Coherence - sub annual and annual statistics

 Not applicable

15.3.2. Coherence - National Accounts

Not applicable

15.4. Coherence - internal

Not applicable


16. Cost and Burden Top
Restricted from publication


17. Data revision Top
17.1. Data revision - policy

To reduce the response burden and the need to contact the enterprises more than necessary, the data should be in as good quality as possible when it received in NSI. Data should be checked by hard and soft controls before they are forwarded to NSI. This will increase data quality and reduce data editing in NSI.

17.2. Data revision - practice

Data are controlled and edited in specific editing tool/software by the statistics responsible of NSI. (please see the details in 18.4)

17.2.1. Data revision - average size

 Not requested


18. Statistical processing Top
18.1. Source data

A) Frame population description and distribution

For more information see concept 18.1.1.

 

B) Sampling design - Sampling method

Description of the sampling method used (e.g. stratified random sample, quota sampling, cluster sampling; one-stage or two-stage sampling) and information which variables were used to stratify, the categories of those variables, in particular for the NACE Rev. 2 categories related to the "possible calculation of European aggregates", and the final number of strata: 

The sample is a stratified random sample. It was drawn from all legal units with at least ten employees and self-employed persons in the Central Register of Establishments and Units. It was stratified by industry and size of the units measured by employment.

Due to the implementation of new NACE breakdowns in 2021, several industries that were belonged to the same breakdown-group earlier have been subdivided into own stand-alone-groups. Some of these industries have relatively few units in their population. There has not been an issue earlier (due to these were belonged to the breakdowns that other industries also are included in the breakdowns) regarding to draw the sample, but since these small industries needed to be broken down into separate groups, hence, there was a need to modify the draw probabilities for these groups. The NSI have therefore examined the target population to identify which industries that have few units regarding the criterion for representative sample. Most of the industries have satisfying units in their population, however, there is an industry that has no unit (in-scope), and a couple industries have less than 20 units in the population. Therefore the NSI have modified the draw probabilities for the industries that have few units, before the stratified random sample. The modification was done in collaboration with experts from the methodological division in SSB.

 The new standardized draw probabilities:

  1. Applies to the NACE-groups with less than 20 employees and self-employed persons in the population: 100%, i.e. full count
  2. Applies to the NACE-groups with 20-999 employees and self-employed persons in the population:
    Among units with 10-19 employees and self-employed persons the sample was 20 per cent, among units with 20-49  employees and self-employed persons the sample was 30 per cent, among units with 50-99 employees and self-employed persons the sample was 50 per cent, among units with 100-249  employees and self-employed persons the sample was 75 per cent and among units with at least 250 employees and self-employed persons all units were included in the sample.
  3. Applies to the NACE-groups with 1000 or more employees and self-employed persons in the population:
    Among units with 10-19 employees and self-employed persons the sample was 7.5 per cent, among units with 20-49  employees and self-employed persons the sample was 15 per cent, among units with 50-99  employees and self-employed persons the sample was 50 per cent, among units with 100-249  employees and self-employed persons the sample was 75 per cent and among units with at least 250 employees and self-employed persons all units were included in the sample.

The difference between 2. and 3. is that there are lower draw probalities for emp.groups 10-19 and 20-49 in 3. compared with 2.

 

C) Gross sample distribution

More detailed information is available in “ Sample and standard error tables 2023  “ excel file (Worksheet: GROSS SAMPLE)

 

D) Net sample distribution

More detailed information is available in “ Sample and standard error tables 2023  “ excel file (Worksheet: NET SAMPLE)



Annexes:
Sample and standard error tables 2023 - NO
18.1.1. Population frame

A) Description of frame population

a) When was the sample for the ICT usage and e-commerce in enterprise survey drawn? January 2023                    
b) When was the last update of the Business register that was used for drawing the sample of enterprises for the survey? January 2023 
c) Please indicate if the frame population is the same as, or is in some way coordinated with, the one used for the Structural Business Statistics (different snapshots) The SBS used a later snapshot, otherwise the same frame definition.
d) Please describe if different frames are used during different stages of the statistical process (e.g. frame used for sampling vs. frame used for grossing up): The same frame is used for sampling and grossing up. 
e) Please indicate shortcomings in terms of timeliness (e.g. time lag between last update of the sampling frame and the moment of the actual sampling), geographical coverage, coverage of different subpopulations, data available etc., and any measures taken to correct it, for this survey. No additional known shortcomings. 

 

 B) Frame population distribution

More detailed information is available in “ Sample and standard error tables 2023  “ excel file (Worksheet: FRAME POPULATION)

18.2. Frequency of data collection

Annual

18.3. Data collection

The survey data was collected via a stand-alone web-survey survey. The model questionnaire was translated by the statistics responsible in cooperation with division for methods and division for enterprise surveys at the NSI. This is done to make sure that the survey questions in Norwegian is easy to understand and hence hard to misinterpret. The translation started in the third quarter 2022. Thereafter, the survey was programmed by survey-programmers from the division for enterprise surveys, based on a technical specification scheme ordered by the statistics responsible, so that the survey includes both mandatory and optional questions with the right filters, and it includes technical functionalities, e.g. automatically controls and checks, explanatory notes, etc. The survey was tested thoroughly by several experts to ensure that there is no errors and technical issue in the web scheme.

The data collection started 13 February 2023. The respondents received an electronic information letter which informs about the survey was available in the Altinn portal, including deadline, and consequences of not responding to the survey. When the deadline was reached, the respondents who have not yet answered the survey will get a warning, and a new deadline. This new deadline will be about ten days later. If this deadline is not met, a compulsory fine will be imposed. Throughout the data collection, the NSI was available for any questions the respondents may have regarding the survey.
Turnover data is collected from the last version from the SBS statistics. Employment and economic activity are collected from administrative register data.

 

See detailed sections below.

18.3.1. Survey period
Survey / Collection Date of sending out questionnaires Date of reception of the last questionnaire treated
General survey  13/02/2023  22/05/2023
Micro-enterprises  Not included  Not included

Micro-enterprises is not covered in the national survey

18.3.2. Survey vehicle – general survey
General survey - Stand-alone survey
18.3.3. Survey vehicle – micro-enterprises
The collection of micro-enterprises was integrated with the general survey
18.3.4. Survey type

Self-administered web questionnaire. 

The web survey was distributed through Altinn, which is an internet portal for digital dialogue between businesses, private individuals and public agencies. Altinn is also a technical platform that government bodies can use to develop digital services.

18.3.5. Survey participation
Mandatory
18.4. Data validation

The server-based validation tool embedded in the eDAMIS4 provided by Eurostat for validation of data was used to validate the data. Due to the implementation of SDMX-format and Regional data in the data set, NSI had pretested the data via the EDAMIS Acceptance portal before transmission. There was no validation error in the data set during the testing. 

Furthermore, the NSI have performed checks of microdata and the aggregated data, at enterprise level and NACE breakdowns. In general, the questionable units or missing variable will be checked, and enterprises will be contacted to confirm the given information was correct. In order to collect high quality raw data, the NSI spent a lot of work on the design of the national survey, e.g. most of the questions are mandatory, technical controls, etc., to limit the non-response and illogical answers, etc. The NSI used their own data revision tool ISEE (Integrated system for editing of data) and other software-based tools to check the micro data and analyse the aggregates. The aggregated data has been compared to national publicized figures, including year-to-year checks, before the submission of the finalized data. There was no treatment for unit non-response and no imputation on missing numerical values.

The validation report from the EDMAIS4 received after the data was transmitted, showed that there was no validation error on the data. The NSI received a notification from EDAMIS4 regarding confirmation of Y2Y check of the indicators at 12th October 2023. The NSI had checked the microdata and edit-tools which was used to construct the variables, no suspicious error has been found. The changes are due to natural causes. The NSI sent an email to Eurostat at 19th October with the confirmed of the results are accurate and reliable.

18.5. Data compilation

Grossing-up procedures

The collected data has been grossed up (weighted). The objective of the weighting is to gross up the units in the sample so that the characteristics of the sample correspond as closely as possible to the population.

The weights were based on the relationship between the sample population and the sample in each separate stratum and take imbalances in non-response into consideration. The grossing-up procedures are based on a breakdown of enterprises in five employment groups (10-19, 20-49, 50-99, 100-249 and 250+ employees and self-employed persons) and the areas of industries in NACE Rev.2 specified by Eurostat. The data material has been weighted to represent the total number of units, turnover and employment in the total population. The grossing-up procedures is performed in collaboration with people from the methodological division in SSB as earlier. The strata will be made up by 32 NACE groups and the 5 size groups. The net sample thus included a total of 160 strata regarding to the presented NACE breakdown of the draft Commission Regulation (EU) specifying the elements of the data to be transmitted for the characteristics for the topic “ICT usage and e-commerce” for reference year 2023 and 2024 in accordance with EC  Regulation.

The 32 NACE groups:

Group 1 = NACE 10-12

Group 2 = NACE 13-15

Group 3 = NACE 16-18

Group 4 = NACE 19 (zero unit in the targeted population)

Group 5 = NACE 20

Group 6 = NACE 21

Group 7 = NACE 22-23

Group 8 = NACE 24-25

Group 9 = NACE 26

Group 10 = NACE 27

Group 11 = NACE 28

Group 12 = NACE 29-30

Group 13 = NACE 31-33

Group 14 = NACE 35

Group 15 = NACE 36-39

Group 16 = NACE 41-43

Group 17 = NACE 45

Group 18 = NACE 46

Group 19 = NACE 47

Group 20 = NACE 49-53

Group 21 = NACE 55

Group 22 = NACE 56

Group 23 = NACE 58-60

Group 24 = NACE 61

Group 25 = NACE 62-63

Group 26 = NACE 68

Group 27 = NACE 69-71

Group 28 = NACE 72
Group 29 = NACE 73-75
Group 30 = NACE 77-78 + 80-82

Group 31= NACE 79
Group 32= NACE 95.1 

18.5.1. Imputation - rate

Not applicable (no need for imputation)

18.6. Adjustment

Not applicable

18.6.1. Seasonal adjustment

Not applicable


19. Comment Top

Problems encountered and lessons to be learnt: 

19.1. Documents
Questionnaire in national language  Bruk av IKT  i næringslivet  2023
Questionnaire in English (if available)  ICT ENT 2023 survey in English
National reports on methodology (if available) Information on methodology can be found in this website (please see "About the statistics") ICT usage in enterprises – SSB
Analysis of key results, backed up by tables and graphs in English (if available)  Please click on the links for the results, tables, etc.

ICT usage in enterprises – SSB

ICT usage in enterprises. Statbank Norway (ssb.no)

Other Annexes ICT ENT 2023 survey in national language


Annexes:
ICT ENT 2023 survey in English
ICT ENT Survey 2023 in national language


Related metadata Top


Annexes Top
Annex I._Completeness 2023
Annex II._ Accuracy 2023
Sample and standard error tables 2023