Statistics Denmark

Research, Technology and Culture

Sejrgade 11, 2100 Kbenhavn, Denmark

The Community Innovation Survey (CIS) is a survey about innovation activities in enterprises.

The CIS provides statistics by type of innovators, economic activities and size class of enterprises. The survey is currently carried out every two years across the EU Member States, EFTA countries and EU candidate countries.

In order to ensure comparability across countries, Eurostat together with the countries developed a Harmonised Data Collection (HDC) questionnaire accompanied by a set of definitions and methodological recommendations.

CIS 2018 concepts and its underlying methodology are based on the Oslo Manual (2018) 4th Edition.

The aim of the statistics is to analyze the scope, type and effect of business enterprise innovation, including shedding light on the innovation activities.

Innovation in the enterprise sector is a yearly statistics on resources used for R&D and share of innovative enterprises. The statistics are distributed by sector, size class and region.

One part of data concerns the inputs, e.g. activities and resources used, a second part concerns the process (conditions and knowledge sharing) and a third part concerns the output.

The most important indicators are:

Share of innovators by type of innovation: - Product innovators - Process innovators - Innovators total

Innovation expenses, excl. research and development (R&D) - Expenses for wages and social contributions to innovation, excl. R&D - Expenses for other running costs for innovation - Purchase of material , equipment and software - Purchase of external rights - Purchase of other external knowledge - Purchase of consultancy services

Share of enterprises with and without cooperation on innovation activities.

Indicators related to the enterprises are classified by country, economic activity (NACE Rev. 2), size class of enterprises and type of innovation.

The main typology of classification of enterprises in reference to innovation is the distinction between innovation-active enterprises (INN) and not innovation-active enterprises (NINN).

The enterprise is considered as innovative (INN) if during the reference period it successfully introduced a product or business process innovation, had ongoing innovation activities, abandoned innovation activities or was engaged in in-house R&D or R&D contracted out. Non-innovative (NINN) enterprises had no innovation activity mentioned above whatsoever during the reference period.

In accordance with Commission Regulation 995/2012 on innovation statistics, the following industries and services are included in the core target population. Results are made available with these following breakdowns :

All NACE – Core NACE (NACE Rev. 2  sections & divisions B-C-D-E-46-H-J-K-71-72-73 )

CORE INDUSTRY (excluding construction) (NACE Rev. 2 SECTIONS B_C_D_E)

10-12: Manufacture of food products, beverages and tobacco

13-15: Manufacture of textiles, wearing apparel, leather and related products

16-18: Manufacture of wood, paper, printing and reproduction

20: Manufacture of chemicals and chemical products

21: Manufacture of basic pharmaceutical products and pharmaceutical preparations

19-22: Manufacture of petroleum, chemical, pharmaceutical, rubber and plastic products

23: Manufacture of other non-metallic mineral products

24: Manufacture of basic metals

25: Manufacture of fabricated metal products, except machinery and equipment

26: Manufacture of computer, electronic and optical products

25-30: Manufacture of fabricated metal products (except machinery and equipment), computer, electronic and optical products, electrical equipment, motor vehicles and other transport equipment

31-33: Manufacture of furniture; jewellery, musical instruments, toys; repair and installation of machinery and equipment

D: ELECTRICITY, GAS, STEAM AND AIR CONDITIONING SUPPLY

E: WATER SUPPLY; SEWERAGE, WASTE MANAGEMENT AND REMEDIATION ACTIVITIES

36: Water collection, treatment and supply

37-39: Sewerage, waste management, remediation activities

CORE SERVICES (NACE Rev. 2 sections & divisions 46-H-J-K-71-72-73)(NACE code in the tables = G46-M73_INN)

46: Wholesale trade, except of motor vehicles and motorcycles

H: TRANSPORTATION AND STORAGE

49-51: Land transport and transport via pipelines, water transport and air transport

52-53: Warehousing and support activities for transportation and postal and courier activities

J: INFORMATION AND COMMUNICATION

58: Publishing activities

61: Telecommunications

62: Computer programming, consultancy and related activities

63: Information service activities

K: FINANCIAL AND INSURANCE ACTIVITIES

64: Financial service activities, except insurance and pension funding

65: Insurance, reinsurance and pension funding, except compulsory social security

66: Activities auxiliary to financial services and insurance activities

M: PROFESSIONAL, SCIENTIFIC AND TECHNICAL ACTIVITIES

71: Architectural and engineering activities; technical testing and analysis

72: Scientific research and development

73: Advertising and market research

71-73: Architectural and engineering activities; technical testing and analysis; Scientific research and development; Advertising and market research

No deviation.

Enterprises in the private business sector are covered.

In accordance with Commission Regulation 995/2012 on innovation statistics, the following size classes of enterprises according to number of employees are included in the core target population of the CIS:

• 10 - 49 employees
• 50 - 249 employees
• 250 or more employees

Size class of enterprise, based on number of full-time equivalents by the following size classes:

• 10-49 full-time employees
• 50-249 full-time employees
• 250(+) full-time employees

The description of concepts, definitions and main statistical variables is available in CIS 2018 European metadata file (ESMS) Results of the community innovation survey 2018 (CIS2018) (inn_cis11) in Eurostat database.

Enterprises (economic units).

Core target population are all enterprises in CORE NACE activities (see 3.3.1) with 10 or more employees.

Denmark

The statistics are distributed by region.

Several rounds of Community Innovation Survey have been conducted so far at two-year interval since end of 90’s.

*two reference periods can be distinguished for CIS light: 2000-2002 and 2001-2003

Not relevant.

CIS indicators are available according to 3 units of measure:

NR: Number for number of enterprises and number of persons employed.

THS_EUR: Thousands of euros. All financial variables are provided in thousands of euros, i.e. Turnover or Innovation expenditure.

PC: Percentage. The percentage is the ratio between the selected combinations of indicators.

For CIS 2018, the time covered by the survey is the 3-year period from the beginning of 2016 to the end of 2018.

Some questions and indicators refer to one year — 2018.

The list of indicators covering the 3-year period and referring to one year according to the HDC is available in the Annex section of the European metadata (ESMS). 

CIS surveys are based on the Commission Regulation No 995/2012, implementing Decision No 1608/2003/EC of the European Parliament and of the Council on the production and development of Community statistics on science and technology.

This Regulation establishes innovation statistics on a statutory basis and makes the delivery of certain variables compulsory e.g. innovation activities, cooperation, development, expenditures and turnover (see the Regulation). Each survey wave may additionally include further variables. 

In addition, the Regulation defines the obligatory cross-coverage of economic sectors and size class of enterprises.

Section 8 of the Act on Statistics Denmark (Consolidated act No 610 of 30 May 2018).

Act on Statistics Denmark (Lov om Danmarks Statistik § 8, jf. Lovbekendtgørelse nr. 610 af 30. maj 2018) https://www.dst.dk/da/OmDS/lovgivning 

Not requested.

CIS data are transmitted to Eurostat via EDAMIS using the secured transmission system.

The description of Data confidentiality policy for Statistics Denmark is available in the annexed document.

Cells are flagged confidential if one or two enterprises dominate the strata.

Preliminary data for 2018 are published 12th February 2020. Final data was published in December 2020.

Scheduled release calendar access via Statistics Denmarks web-side.

Innovation by the business sector (2020) is planned to be published 22-11-2021.

From the next survey, (2020) the statistics is expected to be published app. 11 months after the end of the reference period.

CIS is conducted and disseminated at two-year interval in pair years.

Accessibility and clarity refer to the simplicity and ease for users to access statistics using simple and user-friendly procedure, obtaining them in an expected form and within an acceptable time period, with the appropriate user information and assistance: a global context which finally enables them to make optimum use of the statistics.

See below.

-              Online database (containing all/most results) : https://www.statbank.dk/10149

-              Analytical publication (referring to all/most results) :

-              Analytical publication (referring to specific results, e.g. only for one sector or one specific aspect) : 

Business enterprise innovation is published in Statbank under Innovation in the following tables:

• INN11: Enterprises innovation expenditure by type, type of activity, size class and region
• INN12: Innovative enterprises by industry and sizeclass, region and type of innovation
• INN13: Product innovative enterprises by industry and sizeclass, region and product innovation type
• INN14: Procesinnovative enterprises by industry and sizeclass, region and proces innovation type
• INN15: Enterprises by industry and sizeclass, region and cooperation subject

Not requested.

Data are stored electronically, and micro-data can be used for research purposes. Microdata containing more detailed tables than those published can be provided.

Tables are accessible on Eurostat and OECD's homepages and databases, through which international comparisons can be made.

Not requested.

Colectica Quality Statement.

The statistics completely match the specifications of the EU-regulation and comes up to existing guidelines as the Oslo manual concerning statistics on innovation.

Other documentation on methodology is only available in Danish.

Not requested.

Results from the quality evaluation of products and selected processes are available in detail for each statistics and in summary reports for the Working Group on Quality.

Statistics Denmark follows the recommendations on organisation and management of quality given in the Code of Practice for European Statistics (CoP) and the implementation guidelines given in the Quality Assurance Framework of the European Statistical System (QAF). A Working Group on Quality and a central quality assurance function have been established to continuously carry through control of products and processes.

Statistics Denmark follows the principles in the Code of Practice for European Statistics (CoP) and uses the Quality Assurance Framework of the European Statistical System (QAF) for the implementation of the principles. This involves continuous decentralized and central control of products and processes based on documentation following international standards. The central quality assurance function reports to the Working Group on Quality. Reports include suggestions for improvement that are assessed, decided and subsequently implemented.

As part of the general quality control, a quality handbook has been published for the statistics on R&D and innovation.

The general assessment is that the quality of the Danish Innovation Survey is high. The methodology used is in line with recommandations from the Oslo Manual.

As the survey is based on a sample, uncertainty is attached to all the figures in the form of random variation. This applies, in particular, to the results broken down according to the most detailed industry, region and sizeclass figures, where the figures should only be regarded as normative.

Errors in the data reports and problems for enterprises with determining exact amounts that are used on innovation, and when it is innovation and innovation activities.

The results are used by ministries, organizations, researchers and journalists etc., as a basis for political interventions, analyses, articles etc. It is used for research, publications from ministries and for international comparison. Indicators based on the statistics are included in the EU Innovation Union Scoreboard. The on-going development of the survey contents takes place in close dialogue with national stakeholders as well as in EU fora.

At European level, European Commission users are the principal users of the data and contribute in identifying/defining the topics to be covered. Hence, Eurostat is consulted regularly (at hearings, task forces) to clarify which data is to be compiled.

Eurostat is involved in the process of the development of the model questionnaires at a very early stage. User needs are considered throughout the whole discussion process of the model questionnaires aiming at providing relevant statistical data for monitoring and benchmarking of European policies.

National users: Ministries, organizations, journalists, private enterprises, researchers and students. Statistical Denmark has an on-going dialogue with a variety of national users about the contents and usage of the survey. Ministry of Higher Education and Science is the main national user.

Micro data (anonymized) are made available for the purpose of research.

No systematic information on user satisfaction is collected for this statistics. Primary users are represented in the Contact Group for statistics on Research, Development and Innovation.

The statistics completely matches the specifications of the EU-regulation and comes up to existing guidelines as the Oslo manual concerning statistics on innovation.

Not requested.

The survey is sample based and consequently there is some uncertainty in the results in the form of random variation from the detailed indusrty, region and enterprise size class. Every year uncertainty calculations are produced, and these show that the sampling uncertainty is limited.

The results from the survey for enterprises are based on data from 2957 enterprises from a total population of 9291 enterprises. The overall response rate is 98 percent.

Mesurement error will occur when enterprises do not respond according to the objective reality. This can be due to the respondent's lack of overview of the entire innovation activity in the enterprise.

Accuracy in the statistical sense denotes the closeness of computations or estimates to the exact or true values. Statistics are not equal with the true values because of variability (the statistics change from implementation to implementation of the survey due to random effects) and bias (the average of the possible values of the statistics from implementation to implementation is not equal to the true value due to systematic effects).

That part of the difference between a population value and an estimate thereof, derived from a random sample, which is due to the fact that only a subset of the population is enumerated.

As the survey is based on a sample, uncertainty is attached to all the figures in the form of random variation. This applies, in particular, to the results broken down according to the most detailed industry, region and size class figures, where the figures should only be regarded as normative.

Errors in the data reports and problems for enterprises with determining exact amounts that are used on innovation. 

The results are based on data from 2957 enterprises from a total population of 9291 enterprises. The overall response rate was 98 percent.

The main indicator used to measure sampling errors for CIS data is the coefficient of variation (CV).

Coefficient of Variation= (Square root of the estimate of the sampling variance) / (Estimated value)

Formula:

where

Coefficients of variation (CV) for central indicators in 2018 (preliminary data) are:

- Share of enterprises with innovation: 2.0

- Share of enterprises with product innovation: 3.0

- Share of enterprises with proces innovation: 2.4

- Innovation expenditure, other innovation expenditure: 4.5

Coefficient of variation (%) for key variables by NACE categories and for enterprises with 10 and more employees

[1] = Coefficient of variation for the percentage of innovative enterprises (INN) in the total population of enterprises (ENT18)
[2] = Coefficient of variation for the turnover of product innovative enterprises with new or improved products (TUR_PRD_NEW_MKT), as a percentage of total turnover of product innovative enterprises [TUR18,INNO_PRD].
[3] = Coefficient of variation for percentage of product and/or process innovative enterprises (incl. enterprises with abandoned and or on-going activities) involved in any innovation co-operation arrangement [COOP_ALL,INN], as a percentage of innovative enterprises (INN).

Definition of coefficient of variation:

Coefficient of Variation= (Square root of the estimate of the sampling variance) / (Estimated value).

Formula:

where

Non-sampling errors occur in all phases of a survey. They add to the sampling errors (if present) and contribute to decreasing overall accuracy. It is important to assess their relative weight in the total error and devote appropriate resources for their control and assessment.

Coverage errors (or frame errors) are due to divergences between the target population and the frame population. The frame population is the set of target population members that has a chance to be selected into the survey sample.

There are no observed divergences between the target population and the frame population.

Not requested.

Not requested.

No

No

Measurement errors occur during data collection and generate bias by recording values different than the true ones.

The survey questionnaire used for data collection may have led to the recording of wrong values, or there may be respondent or interviewer bias.

Mesurement error will occur when enterprises do not respond according to the objective reality. This can be due to the respondent's lack of overview of the entire innovation activity in the enterprise.

Data for this statistics are collected via questionnaires for app. 3000 respondents among a population of app. 9300 enterprises. The material is validated already during the response from the enterprise, and afterwards followed by computer-aided validation and manual validation. 

Non sampling relates to unit and item non-response. Non response occurs when a survey fails to collect data on all survey variables from all the population units designated for data collection in a sample or complete enumeration.                                                                                                 

There are two types of non-response:                                                                                                                                                                                                    

1) Unit non-response, which occurs when no data (or so little as to be unusable) are collected about a population unit designated for data collection.                   

a) Un-weighted unit non-response rate (%) = 100*(Number of units with no response or not usable response) / (Total number of in-scope (eligible) units in the sample)                                                                                 

b) Weighted unit non-response rate (%) = 100*(Number of weighted units with no response or not usable response) / (Total number of in-scope (eligible) units in the sample)                                                                                        

2) Item non-response, which occurs when only data on some, but not all survey data items are collected about a population unit designated for data collection.              

a) Un-weighted item non-response rate (%) = 100*(Number of units with no response at all for the item) / (Total number of eligible, for the item, units in the sample i.e. filters have to be taken into account)   

See below.

Un-weighted and weighted unit non-response rate by NACE categories and for enterprises with 10 or more employees

When the sample is drawn a design-weight is calculated. When the data collection is closed a final calibrated weight is calculated, compensating the non-response.

The maximum numbers of recalls/reminders are 5.

See below.

Item non-response rate for Turnover (in Core NACE: B-C-D-E-46-H-J-K-71-72-73 enterprises with 10 or more employees).

Item non-response rate for new questions in CIS t (in Core NACE: B-C-D-E-46-H-J-K-71-72-73 enterprises with 10 or more employees)
 

There are no detected processing errors between data collection and data processing.

Not requested.

Timeliness and punctuality refer to time and dates, but in a different manner.

Preliminary data for 2018 are published 12th February 2020. Final data is expected to be published in November 2020.

Timeliness of national data – date of first release of national level : November 2020

The 2018 statistics are published as preliminary data in February 2020.

From the next survey, (2020) the statistics is expected to be published app. 11 months after the end of the reference period.

Not requested.

Punctuality refers to the time lag between the release date of data and the target date on which they were scheduled for release as announced officially.

Date of transmission of complete and validated data to Eurostat (Number of days between that data and 30 June 2020) : 

Comparability aims at measuring the impact of differences in applied statistical concepts and definitions on the comparison of statistics between geographical areas, non-geographical domains, or over time.

The coherence of statistical outputs refers to the degree to which the statistical processes by which they were generated used the same concepts (classifications, definitions, and target populations) and harmonised methods. Coherent statistical outputs have the potential to be validly combined and used jointly.

The statistics for 2018 is only to a certain degree comparable to results from former surveys. The statistics is to a certain degree comparable with the statistics on innovation in the public sector.

Delivery of data to Eurostat follows the minimum rules laid down in the regulation, which means that the data cover the types of activities and size classes of enterprises, which are defined by the regulation. Thereby the statistics is comparable to the similar statistics of other EU countries for the types of activities and size classes covered by the statistics.

Not requested.

Methodological deviations from the CIS Harmonised Data Collection (HDC)

Methodological deviations from the CIS Harmonised Data Collection (HDC)

Due to important methodological changes in CIS 2018 driven by Oslo Manual 2018, the data 2018 cannot be directly compared with previous CIS waves.

The surveys run between 2007 and 2016 are comparable. The survey for 2018 differs in a range of ways from the former surveys, and results are therefore only to a certain degree comparable to results from former surveys. The changes introduced with the 2018-survey are:

• The coverage of activities and size classes are changed to comply with the minimum rules of the EU regulation. In the previous surveys the NACE coverage of the enterprises was wider than required by the EU Regulation.
• The structure of the questionnaire is changed, new questions are introduced and others have been changed
• The definitions of R&D and innovation have been clarified. E.g. marketing and organizational innovations are no longer considered independent types of innovation, but elements of these former types are included in business process innovation

Not requested.

See the comparison between SBS and CIS data in the section 15.3.3 below.

Not requested.

Not requested.

There are no other comparable Danish statistics. The results can be compared to those of other EU countries, since there is a harmonized methodological foundation.

No differences between CIS data and SBS.

This part compares key variables for aggregated CIS data with SBS data
Definition of relative difference between CIS and SBS data: DIFF = (SBS/CIS)*100

Comparison between SBS and CIS data (relative difference) by NACE categories and for enterprises with 10 or more employees

* Numbers are to be provided for the last year of the reference period (t)

The data are to a large extent consistent, partly as a consequence of the electronic questionnaire guiding the respondents, and partly as a reflection of validation and correction.

Confidential information on the production cost of the CIS.

Statistics Denmark revises published figures in accordance with the Revision Policy for Statistics Denmark. The common procedures and principles of the Revision Policy are for some statistics supplemented by a specific revision practice.

 Not requested.

Not requested.

The statistics are compiled on the basis of questionnaires collected from app. 3,000 enterprises drawn as a sample from a population of app. 9,300 enterprises. The statistics are collected as one part of a single questionnaire, that also covers enterprises' research and development (R&D).

The frame population is drawn from the Business Register, and consists of a population of 9291 enterprises in 2018.

The enterprises are sampled depending on the number of full-time equivalents and type of activity (NACE). All enterprises with 100 or more full-time equivalents are included in the sample, and the likeliness of being chosen for the sample decreases in line with decrease in number of full-time equivalents. The enterprises in the sample are randomly selected.

Variables and indicators filled or prefilled from other sources.

According to the Commission Regulation (UE) 995/2012, the innovation statistics shall be provided to Eurostat every two years in each even year. The data collection takes place every second year in year t-2 preceding the data provision.

The statistics are collected via http://www.virk.dk as an electronic questionnaire.

Mandatory

The statistics are collected via combination of a random survey and census.

http://www.virk.dk as an electronic questionnaire.

Combination of survey and a census

The enterprises in the census are characterized by at least one of the following criterias.

• Reported R&D expenditures of at least 5 mill D.kr in at least one of the two past years 
• Reported innovation expenditures of at least 5 mill D.kr the past year 
• Has 100 or more employees 
• is in the R&D service industry 
• is in the Advanced Technology Group (GTS)

Enterprises in the census all received a questionnaire and therefore they are automatically part of the sample.
The rest of the sample is drawn from the rest of the population.

Data collection method

A comprehensive validation of the data is carried out: In the electronic questionnaire validation is performed on a range of the variables, e.g. on totals. After the data collection the data are mechanically validated and to some extent corrected. The ICT-programs that checks the data for errors also forms lists of likely or de facto errors. The types of errors that are identified as those having the greatest influence on the quality of the statistics are listed together with identification numbers of the respondents and checked manually. Finally outlier tests are carried out for key variables/combinations of these. A minor part of the data collected is compared to other sources with the aim of assessing whether the response is likely correct or should be corrected. This applies to e.g. to the innovation expenses, which are compared to the total turnover of the enterprise, which comes from The Central Business Register. Also public accounts from the enterprises are used as a supplying source of information.

Operations performed on data to derive new information according to a given set of rules.

The final, corrected data material is compared to the original sample. In 2018 no enterprises had their response calculated from imputation. A calibrated weighting is carried out.

In 2018 no enterprises had their response calculated from imputation.

Imputation is the method of creating plausible (but artificial) substitute values for all those missing.

Definition of imputation rate:

Imputation rate (for the variable x) (%) = 100*(Number of replaced values) / (Total number of values for a given variable)

Definition of weighted imputation rate:

Weighted imputation rate= 100*(Number of total weighted replaced values) / (Total number of weighted values for a given variable)

Imputation rate for metric variables by NACE categories and for enterprises with 10 or more employees:

(1) = Total turnover in the last year of the reference period (t) (TUR)

(2) = Share of the turnover in the last year of the reference period (t) due to new or improved product new to the market in the total turnover for product innovative enterprises TUR_PRD_NEW_MKT/TUR(INNO_PRD)

(3) = R&D expenditure performed in-house (EXP_INNO_RND_IH)

Weights calculation method for sample surveys:

The results are raised to 100 percent of the surveyed population based on the stratums used in the sampling procedure. By grossing up a reweighting and calibration using regression techniques is applied to the weight of the individual enterprise. Units nonresponse is handling through reweighting as part of the grossing up procedure. Grossing up is done using CLAN with the assistance of in‐house methodology experts.

Weight = Nstrata - nstrata
The weight is calibrated by number of employees, turnover and region.

Software SAS and CLAN

Calibration via the SAS-macro CLAN (Andersson and Nordberg, 1998). The aim of the calibration is to correct skewness as a result of non-response, via using the knowledge on population level for number of units together with the additional variables turnover and number of full-time equivalents.

No weighting for cut-off.

Not requested.