Statistics Austria

Research and Digitalisation Unit

Directorate Social Statistics

Guglgasse 13

1110 Wien

Austria

The Community Innovation Survey (CIS) is a survey about innovation activities in enterprises. The survey is designed to collect the information on types of innovation, processes of development of innovation like cooperation patterns, financing and expenditure, objectives of innovation activities or barriers for initiating or implementing innovation.

The CIS provides statistics by type of innovators, economic activity and size class of enterprises. The survey is currently carried out every two years across the EU Member States, EFTA countries and EU candidate countries.

In order to ensure comparability across countries, Eurostat together with the countries develops a Harmonised Data Collection (HDC) questionnaire and drafts the methodological recommendations for implementation of each survey round. 

CIS 2020 is a second in a row to implement concepts and methodology of the Oslo Manual 4th Edition revised in 2018. The changes that CIS undergone due to revision of the manual and impact on collected indicators are described in the Statistics Explained article: Community Innovation Survey – new features. 

The legal framework for CIS since 2012 is the Commission Regulation No 995/2012 that establishes the quality conditions and identifies the obligatory cross-coverage of economic sectors, size class of enterprises and innovation indicators. The target population are enterprises with at least 10 employees classified in the core NACE economic sectors (see 3.3).  Further activities may be covered on a voluntary basis in national datasets. Most statistics are based on the 3-year reference period (t, t-1, t-2), but some use only one calendar year (t or t-2). 

Indicators related to the enterprises are classified by country, economic activity (NACE Rev. 2), size class of enterprises and type of innovation.

The main typology of classification of enterprises in reference to innovation is the distinction between innovation-active enterprises (INN) and not innovation-active enterprises (NINN).

The enterprise is considered as innovative (INN) if during the reference period it successfully introduced a a) product or a) business process innovation, c) completed but not yet implemented the innovation, d) had ongoing innovation activities, e) abandoned innovation activities or was f) engaged in in-house R&D or R&D contracted out. Non-innovative (NINN) enterprises had no innovation activity mentioned above whatsoever during the reference period.

CIS covers main economic sectors according to NACE Rev.2 broken down by size class of enterprises and type of innovation activity.

In accordance with Commission Regulation 995/2012 on innovation statistics, the following industries and services are included in the core target population. Results are made available with these following breakdowns :

All NACE – Core NACE (NACE Rev. 2  sections & divisions B-C-D-E-46-H-J-K-71-72-73 )

CORE INDUSTRY (excluding construction) (NACE Rev. 2 SECTIONS B_C_D_E)

10-12: Manufacture of food products, beverages and tobacco

13-15: Manufacture of textiles, wearing apparel, leather and related products

16-18: Manufacture of wood, paper, printing and reproduction

20: Manufacture of chemicals and chemical products

21: Manufacture of basic pharmaceutical products and pharmaceutical preparations

19-22: Manufacture of petroleum, chemical, pharmaceutical, rubber and plastic products

23: Manufacture of other non-metallic mineral products

24: Manufacture of basic metals

25: Manufacture of fabricated metal products, except machinery and equipment

26: Manufacture of computer, electronic and optical products

25-30: Manufacture of fabricated metal products (except machinery and equipment), computer, electronic and optical products, electrical equipment, motor vehicles and other transport equipment

31-33: Manufacture of furniture; jewellery, musical instruments, toys; repair and installation of machinery and equipment

D: ELECTRICITY, GAS, STEAM AND AIR CONDITIONING SUPPLY

E: WATER SUPPLY; SEWERAGE, WASTE MANAGEMENT AND REMEDIATION ACTIVITIES

36: Water collection, treatment and supply

37-39: Sewerage, waste management, remediation activities

CORE SERVICES (NACE Rev. 2 sections & divisions 46-H-J-K-71-72-73)(NACE code in the tables = G46-M73_INN)

46: Wholesale trade, except of motor vehicles and motorcycles

H: TRANSPORTATION AND STORAGE

49-51: Land transport and transport via pipelines, water transport and air transport

52-53: Warehousing and support activities for transportation and postal and courier activities

J: INFORMATION AND COMMUNICATION

58: Publishing activities

61: Telecommunications

62: Computer programming, consultancy and related activities

63: Information service activities

K: FINANCIAL AND INSURANCE ACTIVITIES

64: Financial service activities, except insurance and pension funding

65: Insurance, reinsurance and pension funding, except compulsory social security

66: Activities auxiliary to financial services and insurance activities

M: PROFESSIONAL, SCIENTIFIC AND TECHNICAL ACTIVITIES

71: Architectural and engineering activities; technical testing and analysis

72: Scientific research and development

73: Advertising and market research

71-73: Architectural and engineering activities; technical testing and analysis; Scientific research and development; Advertising and market research

All these industries were covered in the survey. No further industries were covered in addition.

In accordance with Commission Regulation 995/2012 on innovation statistics, the following size classes of enterprises according to number of employees are included in the core target population of the CIS:

• 10 - 49 employees
• 50 - 249 employees
• 250 or more employees

No national differences regarding sector and industry.

Size classes were based on persons employed.

The description of concepts, definitions and main statistical variables is available in CIS 2020 European metadata file (ESMS) Results of the community innovation survey 2020 (CIS2020) (inn_cis12) in Eurostat database.

Statistical unit and reporting unit is the “legal unit” as in all previous CIS surveys

Core target population are all enterprises in CORE NACE activities (see 3.3.1) with 10 or more persons employed.

The entire country is covered. No regional data are published, but it is foreseen to provide Eurostat with information on NUTS 1 level for selected indicators.

Several rounds of Community Innovation Survey have been conducted so far at two-year interval since end of 90’s.

*two reference periods can be distinguished for CIS light: 2000-2002 and 2001-2003

Not relevant.

CIS indicators are available according to 3 units of measure:

NR: Number for number of enterprises and number of persons employed.

THS_EUR: Thousands of euros. All financial variables are provided in thousands of euros, i.e. Turnover or Innovation expenditure.

PC: Percentage. The percentage is the ratio between the selected combinations of indicators.

For CIS 2020, the time covered by the survey is the 3-year period from the beginning of 2018 to the end of 2020.

Some questions and indicators refer to one year — 2020.

The list of indicators covering the 3-year period and referring to one year according to the HDC is available in the Annex section of the European metadata (ESMS). 

CIS surveys are based on the Commission Regulation No 995/2012, implementing Decision No 1608/2003/EC of the European Parliament and of the Council on the production and development of Community statistics on science and technology.

This Regulation establishes innovation statistics on a statutory basis and makes the delivery of certain variables compulsory e.g. innovation activities, cooperation, development, expenditures and turnover (see the Regulation). Each survey wave may additionally include further variables. 

In addition, the Regulation defines the obligatory cross-coverage of economic sectors and size class of enterprises.

According to the Federal Statistics Law (Bundesstatistikgesetz) Statistics Austria is the National Statistical Institute, responsible for Austrian official statistics. For the CIS 2020 there is no specific legal act. The survey is carried out based on a contract between Statistics Austria and the Federal Ministry for Digital and Economic affairs (BMDW). 

Not requested.

 CIS data are transmitted to Eurostat via EDAMIS using the secured transmission system.

Federal Statistics Law (Bundesstatistikgesetz) requires data to be published that no conclusions can be drawn on the underlying individual micro-data. In business surveys this is implemented in a way that all cells with less than 3 observations (<3) are suppressed.

All cells with a value of less than 3 were sent to Eurostat, but flagged as confidential. Secondary confidentiality was applied afterwards.

National publication of CIS 2020 results in a press release and on the Internet with approx. 10 selected tables: 7 July 2022

Press release is announced in the release calendar on Statistics Austria’s website.

In October 2022 an extensive publication with approx. 240 pages was published on Statistics Austria's website.

https://www.statistik.at/en/medien/release-calendar

Data releases are announced in the official “release calendar” on Statistics Austria’s website. Data releases can have several forms: press conferences, press releases, tables on the website, written reports or a mix of those means. Usually all users are treated equally and receive information at the same time. In exceptional cases, for highly important statistics, this rule might be suspended when the Federal Chancellary ("Prime Minister´s Office") can be informed shortly beforehand (one day before); in such cases, this is publicly announced.

CIS is conducted and disseminated at two-year interval in pair years.

Accessibility and clarity refer to the simplicity and ease for users to access statistics using simple and user-friendly procedure, obtaining them in an expected form and within an acceptable time period, with the appropriate user information and assistance: a global context which finally enables them to make optimum use of the statistics.

See below.

-             Analytical publication (referring to allresults) : 240 pages, released in October 2022, freely available to the public: https://www.statistik.at/en/statistics/research-innovation-digitalisation/innovation-in-enterprises

-             Main tables on the website of Statistics Austria: https://www.statistik.at/en/statistics/research-innovation-digitalisation/innovation-in-enterprises

Online database Statcube (containing all/most results) : https://www.statistik.at/en/databases/statcube/statcube-statistical-database/login 

Not requested.

Microdata is available to researchers via remote access according to the rules of the Austrian Microdata Center (AMDC): https://www.statistik.at/services/tools/services/amdc-mikrodaten-fuer-die-wissenschaft/remote-access-amdc

No other means of dissemination used.

Not requested.

On Statistics Austrias web site, there is also a national standard documentation (Standarddokumentation) available, similar to this report, which also addresses issues of interest for data users (in German only). https://www.statistik.at/statistiken/forschung-innovation-digitalisierung/innovation-in-unternehmen ("Dokumentationen" / "Standarddokumentationen")

Only the Executive Summary is available in English: https://www.statistik.at/en/statistics/research-innovation-digitalisation/innovation-in-enterprises ("Documentation" / "Standard documentation")

Not requested.

The feedback from users is generally positive. In addition to the available metadata as described before, users can contact the experts at Statistics Austria for clarifications and questions.

The specific national quality report (“Standarddokumentation”) is an equivalent to this quality report and available on Statistics Austria’s website (in German only).

Survey is done by highly qualified staff with expertise in innovation statistics. Sample is drawn from the national business register. The web questionnaire contains a large number of automatic plausibility checks. Two written reminders are sent to enterprises. The second reminder contained a printed paper questionnaire to encourage firms to participate in the survey. Enterprises are re-contacted when missing or implausible data are reported. A non-response survey is carried out among the non-responding enterprises. After the data collection another round of plausibility checks is carried out.

Given the preconditions the quality of the CIS data is considered as good. Innovation is a complex concept and difficult to objectify. Innovation expenditures, apart from R&D expenditures, are difficult to quantify.

A relatively high response rate could be reached despite the voluntary character of the survey. The web questionnaire is implemented as the main medium of data reporting and by re-contacting enterprises missing data could be completed and implausible data corrected. Item non-response rates are therefore relatively low.

Relevance is the degree to which statistics meet current and potential users needs. It includes the production of all needed statistics and the extent to which concepts used (definitions, classifications etc.) reflect user needs. The aim is to describe the extent to which the statistics are useful to, and used by, the broadest array of users. For this purpose, statisticians need to compile information, firstly about their users and their needs.

The CIS is based on a common questionnaire and a common survey methodology, as laid down in the 4th edition of Oslo Manual (2018 edition), in order to achieve comparable, harmonised and high quality results for EU Member States, EFTA countries, Candidates and Associated countries.

The decision which questions from the Eurostat Harmonised Data Collection are included in the national survey is taken in close cooperation with the responsible ministry (BMDW: Federal Ministry for Digital and Economic affairs). During this interaction the BMDW expresses its user needs.

The so called "Fachbeiräte" are advisory councils which meet once a every year for each statistical field. The council for Science and technology Statistics ("Fachbeirat für Wissenschafts- und Technologiestatistik") is responsible for innovation statistics. It consists of the main institutional users (federal ministries, regional governments, research institutes, chamber of commerce etc) and can give advice and feedback on these statistics.

No user satisfaction survey was carried out. 

All compulsory data according to the respective EU regulation was transmitted to Eurostat, and a large number of voluntary data.

Not requested.

Accuracy in the statistical sense denotes the closeness of computations or estimates to the exact or true values. Statistics are not equal with the true values because of variability (the statistics change from implementation to implementation of the survey due to random effects) and bias (the average of the possible values of the statistics from implementation to implementation is not equal to the true value due to systematic effects).

That part of the difference between a population value and an estimate thereof, derived from a random sample, which is due to the fact that only a subset of the population is enumerated.

The main indicator used to measure sampling errors for CIS data is the coefficient of variation (CV).

Coefficient of Variation= (Square root of the estimate of the sampling variance) / (Estimated value)

Formula:

where

Coefficient of variation (%) for key variables by NACE categories and for enterprises with 10 and more employees

[1] = Coefficient of variation for the percentage of innovative enterprises (INN) in the total population of enterprises (ENT20)
[2] = Coefficient of variation for the turnover of product innovative enterprises with new or improved products (TUR_PRD_NEW_MKT), as a percentage of total turnover of product innovative enterprises [TUR20,INNO_PRD].
[3] = Coefficient of variation for percentage of product and/or process innovative enterprises (incl. enterprises with abandoned and or on-going activities) involved in any innovation co-operation arrangement [COOP_ALL,INN], as a percentage of innovative enterprises (INN).

The estimation was carried out with the R package "survey".

Non-sampling errors occur in all phases of a survey. They add to the sampling errors (if present) and contribute to decreasing overall accuracy. It is important to assess their relative weight in the total error and devote appropriate resources for their control and assessment.

Coverage errors (or frame errors) are due to divergences between the target population and the frame population. The frame population is the set of target population members that has a chance to be selected into the survey sample. It is a listing of all items in the population from which the sample is drawn that contains contact details as well as sufficient information to perform stratification and sampling.

Not requested.

Not requested.

There is no indication of any under-coverage or over-coverage in the frame population as the official business register is used for sampling.

No

Measurement errors occur during data collection and generate bias by recording values different than the true ones. The survey questionnaire used for data collection may have led to the recording of wrong values, or there may be respondent or interviewer bias.

The Eurostat Harmonised Data Collection was followed closely, and data collection was done by staff from the national statistical office with a very good knowledge of the Oslo Manual innovation concept. A large number of re-contacts with enterprises were made to detect any measurement errors on the respondents' side. Other available data sources, such as R&D data and SBS data were used to cross-check the plausibility of the reported data. In the web questionnaire, a large number of plausibility checks were implemented which warned the respondent when wrong or implausible data were reported.

Non response occurs when a survey fails to collect data on all survey variables from all the population units designated for data collection in a sample or complete enumeration.

There are two types of non-response:

1) Unit non-response, which occurs when no data (or so little as to be unusable) are collected about a population unit designated for data collection.

a) Un-weighted unit non-response rate (%) = 100*(Number of units with no response or not usable response) / (Total number of in-scope (eligible) units in the sample)

b) Weighted unit non-response rate (%) = 100*(Number of weighted units with no response or not usable response) / (Total number of in-scope (eligible) units in the sample)

2) Item non-response, which occurs when only data on some, but not all survey data items are collected about a population unit designated for data collection.

a) Un-weighted item non-response rate (%) = 100*(Number of units with no response at all for the item) / (Total number of eligible, for the item, units in the sample i.e. filters have to be taken into account)

A non-response survey was carried out among 50% of all non-responding firms (=1,600 enterprises). A random sample, stratified by size - 3 classes - and NACE - 3 classes (manufacturing, wholesale, other services) - was drawn. The following questions were asked:

1) During the three years 2018-2020, did your enterprise introduce new or improved goods or services onto the market? (The simple resale of new goods and solely aesthetically changed products are to be excluded) (Yes / No)

2) During the three years 2018-2020, did your enterprise introduce new or improved business processes which differed significantly from the ones used previously? (Yes / No)

This includes: New or improved...
...methods for producing goods or services (including methods for developing goods or services)
...logistics, delivery or distribution methods
...methods for information processing or communication

...methods for accounting or administration
...business practices for organising procedures or external relations
...methods of organising work responsibility, decision making or human resouce management

...marketing methods for promotion, packaging, pricing, product placement or after sales services

3) During the three years 2016-2018, did your enterprise conduct research and development activities (R&D) internally or externally (R&D purchases from third parties)? (Yes / No)
 
The selected enterprises were sent the mini-questionnaire by postal mail and asked to answer the three “Yes/No” questions within two week’s time. Enterprises were asked to return the questionnaire by mail (prepaid envelope was attached), by e-mail, or by fax. 30% of the enterprises (487 units) responded to the survey. 

71% of all enterprises which sent in the mini-questionnaire declared having had at least one of the three types of innovation activity in the reference period. In the course of the regular survey, 61% of all respondents were innovation active. The analysis showed that there was a significant difference between responding and non-responding firms in the CIS 2020. Therefore weights were recalibrated. However, there remains a certain scepticism about the quality of the results of the non-response survey which, inevitably, simplifies extremely complex issues and quality of the results should generally be taken with caution.

See below.

Un-weighted and weighted unit non-response rate by NACE categories and for enterprises with 10 or more persons employed

The number of eligible units is the number of sample units, which indeed belong to the target population.

In total three letters were sent to enterprises, before they were considered non-respondents: One letter informing about the launch of the survey and two additional reminders. The seccond reminder included a paper questionnaire to encourage survey participation.

See below.

Item non-response rate for Turnover (in Core NACE: B-C-D-E-46-H-J-K-71-72-73 enterprises with 10 or more persons employed).

Item non-response rate for new questions in CIS t (in Core NACE: B-C-D-E-46-H-J-K-71-72-73 enterprises with 10 or more employees)
 

 No processing errors are known.  Each step in data processing is usually double-checked. 

Not requested.

Timeliness and punctuality refer to time and dates, but in a different manner.

The timeliness of statistics reflects the length of time between data availability and the event or phenomenon they describe.

Timeliness of national data – date of first release of national level : 7 July 2022

Not requested.

Punctuality refers to the time lag between the release date of data and the target date on which they were scheduled for release as announced officially.

Date of transmission of complete and validated data to Eurostat (Number of days between that data and 30 June 2022) : - 9 days (data were transmitted on 21 June 2022)

Comparability aims at measuring the impact of differences in applied statistical concepts and definitions on the comparison of statistics between geographical areas, non-geographical domains, or over time.

The coherence of statistical outputs refers to the degree to which the statistical processes by which they were generated used the same concepts (classifications, definitions, and target populations) and harmonised methods. Coherent statistical outputs have the potential to be validly combined and used jointly.

Regional data are not published, with the exception of NUTS1 data for specific main indicators which are sent to Eurostat. No comparabilty problem exists. 

Not requested.

Methodological deviations from the CIS Harmonised Data Collection (HDC)

Methodological deviations from the CIS Harmonised Data Collection (HDC)

Due to important methodological changes driven by Oslo Manual 2018, CIS 2016 and CIS 2018 cannot be directly compared with previous CIS waves.

Not requested.

See the comparison between SBS and CIS data in the section 15.3.3 below.

Not requested.

Not requested.

This part compares key variables for aggregated CIS data with SBS data
Definition of relative difference between CIS and SBS data: DIFF = (SBS/CIS)*100

Comparison between SBS and CIS data (relative difference) by NACE categories and for enterprises with 10 or more persons employed

* Numbers are to be provided for the last year of the reference period (t)

Not requested.

Confidential information on the production cost of the CIS.

Not requested.

Not requested.

Not requested.

See below:

The Unternehmensregister (national business register) of Statistics Austria was used for sampling. The register provides name, address, NACE and NUTS classification, number of employees, turnover and information on other characteristics of the enterprise.

A sample of 5,800 enterprises was drawn. Enterprises with 250 and more employed persons were fully enumerated, 50% of those with 50-249 employed persons, and 23% of enterprises with 10 to 49 employed persons. Stratification variables were NACE classes (19 strata), size class (3 strata) and NUTS2 region (9 strata). Altogether 364 strata were were filled with at least one enterprise.

Variables and indicators filled or prefilled from other sources.

According to the Commission Regulation (UE) 995/2012, the innovation statistics shall be provided to Eurostat every two years in each even year t+18. 

The CIS 2020 survey was started on 11 November 2021 by sending out letters to 5,800 enterprises requesting them to send in their data via web questionnaire until 6 December 2021, On 13 December 2021 a reminder was sent out, extending the deadline by 18 January 2022. On 22 January 2022 another reminder was sent out urging to deliver data until 25 February 2022. 45% of the enterprises responded. 80% of all respondents used the web questionnaire, 20% a paper questionnaire. The non-response survey was started on 11 March 2022.

Participation in the survey was voluntary.

A combination of both types was used. Enterprises with 250 and more employed persons were subject to a census (all of them were sampled); small and medium enterprises were selected via a stratified random sample as described above. Mainly web questionnaire with an option to report on paper.

Enterprises with 250 and more employed persons of all NACE classes were subject to a census; small and medium enterprises (less than 250 employed persons, but more than 9) were selected via a stratified random sample. Micro-enterprises with less than 10 employed persons were not surveyed.

Size class. Enterprises with 250 and more employed persons were subject to a census.

 Data collection method

Not requested.

Operations performed on data to derive new information according to a given set of rules.

Imputation is the method of creating plausible (but artificial) substitute values for all those missing.

Definition of imputation rate:

Imputation rate (for the variable x) (%) = 100*(Number of replaced values) / (Total number of values for a given variable)

Definition of weighted imputation rate:

Weighted imputation rate= 100*(Number of total weighted replaced values) / (Total number of weighted values for a given variable)

Imputation rate for metric variables by NACE categories and for enterprises with 10 or more persons employed:

(1) = Total turnover in the last year of the reference period (t) (TUR)

(2) = Share of the turnover in the last year of the reference period (t) due to new or improved product new to the market in the total turnover for product innovative enterprises TUR_PRD_NEW_MKT/TUR(INNO_PRD)

(3) = R&D expenditure performed in-house (EXP_INNO_RND_IH)

Total turnover (1): Data was taken from SBS 2020 which is a compulsory survey. Imputation rates in SBS are not known, but considered very small.

R&D expenditure in-house (3): Enterprises reporting in-house R&D activities, but do no report in-house R&D expenditure (i.e. item non-response, where 0 is an acceptable answer) were imputed with their own R&D expenditure from 2019. No imputation rates are available, but presumably small.

Weights calculation method for sample surveys

Iterative proportional fitting was used to adjust the weights according to the number of enterprises in the strata and the number of innovative enterprises by size class. A self-developed SAS macro was used.

Not requested.