1.1. Contact organisation
Belgian Science Policy Office (BELSPO)
1.2. Contact organisation unit
MERI
1.3. Contact name
Confidential because of GDPR
1.4. Contact person function
Confidential because of GDPR
1.5. Contact mail address
Belspo
WTCIII
Simon Bolivarlaan 30/7
1000 Brussels
Belgium
1.6. Contact email address
Confidential because of GDPR
1.7. Contact phone number
Confidential because of GDPR
1.8. Contact fax number
Confidential because of GDPR
1 July 2020
2.1. Metadata last certified
1 July 2021
2.2. Metadata last posted
27 May 2024
2.3. Metadata last update
27 May 2024
3.1. Data description
The Community Innovation Survey (CIS) is a survey about innovation activities in enterprises. The survey is designed to collect information on types of innovation, processes of development of innovation like cooperation patterns, financing and expenditure, objectives of innovation activities or barriers for initiating or implementing innovation.
The CIS provides statistics by type of innovators, economic activity, and size class of enterprises. The survey is currently carried out every two years across the EU Member States, EFTA countries and EU candidate countries.
In order to ensure comparability across countries, Eurostat together with the member countries develops a Harmonised Data Collection (HDC) questionnaire and drafts the methodological recommendations for implementation of each survey round.
CIS 2020 is a second in a row to implement concepts and methodology of the Oslo Manual 4th Edition revised in 2018. Please refer to the Annex section of the European metadata (ESMS) for details of the time coverage of collected indicators. Community Innovation Survey – new features.
The legal framework for CIS since 2012 is the Commission Regulation No 995/2012 that establishes the quality conditions for the data collection and transmission and identifies the obligatory cross-coverage of economic sectors, size class of enterprises and innovation indicators. The target population are enterprises with at least 10 employees classified in the core NACE economic sectors (see 3.3). Further activities may be covered on a voluntary basis in national datasets. Most statistics are based on the 3-year reference period (t, t-1, t-2), but some use only one calendar year (t or t-2). Please refer to the Annex section of the European metadata (ESMS) for details of the time coverage of collected indicators.
3.2. Classification system
Indicators related to the enterprises are classified by country, economic activity (NACE Rev. 2), size class of enterprises, and type of innovation.
The main typology of classification of enterprises in reference to innovation is the distinction between innovation-active enterprises (INN) and not innovation-active enterprises (NINN).
The enterprise is considered as innovative (INN) if during the reference period it successfully introduced a a) product or a) business process innovation, c) completed but not yet implemented the innovation, d) had ongoing innovation activities, e) abandoned innovation activities or was f) engaged in in-house R&D or R&D contracted out. Non-innovative (NINN) enterprises had no innovation activity mentioned above whatsoever during the reference period.
3.3. Coverage - sector
CIS covers main economic sectors according to NACE Rev.2 broken down by size class of enterprises and type of innovation activity.
3.3.1. Main economic sectors covered - NACE Rev.2
In accordance with Commission Regulation 995/2012 on innovation statistics, the following industries and services are included in the core target population. Results are made available with these following breakdowns :
All NACE – Core NACE (NACE Rev. 2 sections & divisions B-C-D-E-46-H-J-K-71-72-73 )
CORE INDUSTRY (excluding construction) (NACE Rev. 2 SECTIONS B_C_D_E)
10-12: Manufacture of food products, beverages and tobacco
13-15: Manufacture of textiles, wearing apparel, leather and related products
16-18: Manufacture of wood, paper, printing and reproduction
20: Manufacture of chemicals and chemical products
21: Manufacture of basic pharmaceutical products and pharmaceutical preparations
19-22: Manufacture of petroleum, chemical, pharmaceutical, rubber and plastic products
23: Manufacture of other non-metallic mineral products
24: Manufacture of basic metals
25: Manufacture of fabricated metal products, except machinery and equipment
26: Manufacture of computer, electronic and optical products
25-30: Manufacture of fabricated metal products (except machinery and equipment), computer, electronic and optical products, electrical equipment, motor vehicles and other transport equipment
31-33: Manufacture of furniture; jewellery, musical instruments, toys; repair and installation of machinery and equipment
D: ELECTRICITY, GAS, STEAM AND AIR CONDITIONING SUPPLY
E: WATER SUPPLY; SEWERAGE, WASTE MANAGEMENT AND REMEDIATION ACTIVITIES
36: Water collection, treatment and supply
37-39: Sewerage, waste management, remediation activities
CORE SERVICES (NACE Rev. 2 sections & divisions 46-H-J-K-71-72-73)(NACE code in the tables = G46-M73_INN)
46: Wholesale trade, except of motor vehicles and motorcycles
H: TRANSPORTATION AND STORAGE
49-51: Land transport and transport via pipelines, water transport and air transport
52-53: Warehousing and support activities for transportation and postal and courier activities
J: INFORMATION AND COMMUNICATION
58: Publishing activities
61: Telecommunications
62: Computer programming, consultancy and related activities
63: Information service activities
K: FINANCIAL AND INSURANCE ACTIVITIES
64: Financial service activities, except insurance and pension funding
65: Insurance, reinsurance and pension funding, except compulsory social security
66: Activities auxiliary to financial services and insurance activities
M: PROFESSIONAL, SCIENTIFIC AND TECHNICAL ACTIVITIES
71: Architectural and engineering activities; technical testing and analysis
72: Scientific research and development
73: Advertising and market research
71-73: Architectural and engineering activities; technical testing and analysis; Scientific research and development; Advertising and market research
3.3.1.1. Main economic sectors covered - NACE Rev.2 - national particularities
There were no deviations
3.3.2. Sector coverage - size class
In accordance with Commission Regulation 995/2012 on innovation statistics, the following size classes of enterprises according to number of employees are included in the core target population of the CIS:
- 10 - 49 employees
- 50 - 249 employees
- 250 or more employees
3.3.2.1. Sector coverage - size class - national particularities
We used the three size classes provided above, based on the number of employees.
3.4. Statistical concepts and definitions
The description of concepts, definitions and main statistical variables is available in CIS 2020 European metadata file (ESMS) Results of the community innovation survey 2020 (CIS2020) (inn_cis12) in Eurostat database.
3.5. Statistical unit
The legal unit was used as the statistical unit.
3.6. Statistical population
Core target population are all enterprises in CORE NACE activities (see 3.3.1) with 10 or more employees.
3.7. Reference area
Belgium is composed of 3 regions (at NUTS 1 level): Brussels, Flanders, and Wallonia. Each region is endowed with its own statistical office. Thus, three separate samples were drawn.
3.8. Coverage - Time
Several rounds of Community Innovation Survey have been conducted so far at two-year interval since end of 90’s.
3.8.1. Participation in the CIS waves
| CIS wave | Reference period | Participation | Comment (deviation from reference period) |
| CIS2 | 1994-1996 | Yes | No |
| CIS3 | 1998-2000 | Yes | No |
| CIS light | 2002-2003* | No | |
| CIS4 | 2002-2004 | Yes | No |
| CIS2006 | 2004-2006 | Yes | No |
| CIS2008 | 2006-2008 | Yes | No |
| CIS2010 | 2008-2010 | Yes | No |
| CIS2012 | 2010-2012 | Yes | No |
| CIS2014 | 2012-2014 | Yes | No |
| CIS2016 | 2014-2016 | Yes | No |
| CIS2018 | 2016-2018 | Yes | No |
| CIS2020 | 2018-2020 | Yes | No |
*two reference periods can be distinguished for CIS light: 2000-2002 and 2001-2003
3.9. Base period
Not relevant.
CIS indicators are available according to 3 units of measure:
NR: Number for number of enterprises and number of persons employed.
THS_EUR: Thousands of euros. All financial variables are provided in thousands of euros, i.e. Turnover or Innovation expenditure.
PC: Percentage. The percentage is the ratio between the selected combinations of indicators.
For CIS 2020, the time covered by the survey is the 3-year period from the beginning of 2018 to the end of 2020.
Some questions and indicators refer to one year — 2020.
The list of indicators covering the 3-year period and referring to one year according to the HDC is available in the Annex section of the European metadata (ESMS).
6.1. Institutional Mandate - legal acts and other agreements
CIS surveys are based on the Commission Regulation No 995/2012, implementing Decision No 1608/2003/EC of the European Parliament and of the Council on the production and development of Community statistics on science and technology.
This Regulation establishes innovation statistics on a statutory basis and makes the delivery of certain variables compulsory e.g. innovation activities, cooperation, development, expenditures and turnover (see the Regulation). Each survey wave may additionally include further variables.
In addition, the Regulation defines the obligatory cross-coverage of economic sectors and size class of enterprises.
6.1.1. National legislation
Legal agreement on how the three regions and the national government organize the production of official STI statistics:
http://www.ejustice.just.fgov.be/mopdf/2020/01/21_1.pdf#Page195
6.2. Institutional Mandate - data sharing
Not requested.
CIS data are transmitted to Eurostat via EDAMIS using the secured transmission system.
7.1. Confidentiality - policy
The Belgian Science Policy Office (Belspo) is recognized as an ONA (other national authority), i.e. a body (that is not the national statistical office) within the European Statistical System responsible for the production of one or more European statistics. (https://ec.europa.eu/eurostat/web/european-statistical-system a list of all ONA's and notes on their rights and obligations). In Belgium, the production of innovation statistics falls under the three region's responsibility. Through a cooperation agreement (16/4/2006) between regions and the national government, Belgium organizes the production of official STI statistics, stipulating that Belgium will follow Eurostat and OECD methodology. R&D and innovation data are produced within this framework.
Furthermore, the Belgian Interfederal Institute of Statistics (IIS) was created on the basis of a Cooperation Agreement (15/07/2014) (see https://www.iis-statistics.be/index_fr.html). It coordinates the statistics production at the regional and national level in Belgium. It abides by the European Statistics Code of Practice, including its principle 5, on statistical confidentiality and data protection (https://www.iis-statistics.be/doc/CoC_fr.pdf).
7.2. Confidentiality - data treatment
Cells were flagged as confidential (“C”) if the population for that cell (or a set of interdependent cells) contained 10 or fewer enterprises.
8.1. Release calendar
For Statistics Flanders: https://www.vlaanderen.be/statistiek-vlaanderen/publicatieagenda The publication date and the date when the next update will be made available is also published next to each individual statistic.
8.2. Release calendar access
For Statistics Flanders: https://www.vlaanderen.be/statistiek-vlaanderen/publicatieagenda The publication date and the date when the next update will be made available is also published next to each individual statistic.
8.3. Release policy - user access
In Flanders, two core R&D statistics based on CIS are published online on June 30. More general innovation statistics derived from CIS are published online on September 30.
In Belgium, academic researchers may obtain access to regional microdata for academic purposes by signing confidentiality agreements. They may obtain access to the national microdata by submitting a project proposal to a committee that represents both the national and the regional levels, and by signing a confidentiality agreement.
The Belgian Science Policy Office publishes results for the CIS on its website (meri.belspo.be) by the end of the year data were transmitted to Eurostat. An interactive tool is available, as well as a short document providing some (possible) explanations and background for the most salient results.
CIS is conducted and disseminated at a two-year interval in even years.
Accessibility and clarity refer to the simplicity and ease for users to access statistics using simple and user-friendly procedure, obtaining them in an expected form and within an acceptable time period, with the appropriate user information and assistance: a global context which finally enables them to make optimum use of the statistics.
10.1. Dissemination format - News release
See below.
10.1.1. Availability of the releases
| Dissemination and access | Availability | Comments, links, ... |
| Press release | No | |
| Access to public free of charge | Yes | Website with official statistics: https://meri.belspo.be/site/index_en.stm which allows regional comparisons. The last update for this website will be October 2022. |
| Access to public restricted (membership/password/part of data provided, etc) | Yes | Microdata only available to academic researchers, after having signed a confidentiality agreement |
10.2. Dissemination format - Publications
- Online database (containing all/most results and commentary) : https://meri.belspo.be/site/index_en.stm
- Analytical publication (referring to all/most results) : Regional results: https://www.ecoom.be/nodes/cisrapport/en; https://www.vlaamsindicatorenboek.be/4.4/innovatie-inspanningen-van-ondernemingen; https://recherche.wallonie.be/go/INNO-fr; https://meri.belspo.be/site/index_en.stm
- Analytical publication (referring to specific results, e.g. only for one sector or one specific aspect) : Core R&D numbers in Flanders, to monitor the progress towards the 3% target for R&D: https://www.ecoom.be/en/services/3note; https://www.statistiekvlaanderen.be/en/rd-intensity-0; https://www.statistiekvlaanderen.be/en/rd-personnel-0; https://www.vlaamsindicatorenboek.be/2.1/totale-oo-uitgaven-gerd
10.3. Dissemination format - online database
Tabulated data are available online with some degree of aggregation, both at national and regional level.
https://meri.belspo.be/site/index_en.stm
10.3.1. Data tables - consultations
Not requested.
10.4. Dissemination format - microdata access
Microdata is available at the Eurostat Safe center. Upon request, microdata can be made available if all three regions have agreed and the user has signed a standard confidentiality agreement statement (adherence to the laws governing statistical practices).
10.4.1. Dissemination of microdata
| Mean of dissemination | Availability of microdata | Comments, links, ... |
| Eurostat SAFE centre | Yes | |
| National SAFE centre | Yes | Upon request, academic researchers can obtain microdata either from a region for regional data, or from Belspo for national microdata as long as confidentiality agreements are signed. At the national level, data are usually delivered without business enterprise ID’s, but are generally not altered nor recoded |
| Eurostat: partially anonymised data (SUF) | No | |
| National : partially anonymised data | No |
10.5. Dissemination format - other
None
10.5.1. Metadata - consultations
Not requested.
10.6. Documentation on methodology
https://meri.belspo.be/site/index_en.stm provides some meta-data. Quality Reports can be delivered upon request, but are not generally sought after. Starting with CIS 2020, counrtries' integrated metadata and quality reports are made availabe in the EUROSTAT database.
10.6.1. Metadata completeness - rate
Not requested.
10.7. Quality management - documentation
In our publications, we refer to the international guidelines (EUROSTAT and OECD (e.g., Oslo Manual)) which we follow. If quality reports were requested, they would be made available to users, but this hasn't happened yet.
11.1. Quality assurance
Belgium follows Eurostat's recommendations.
11.2. Quality management - assessment
The fact that the CIS is not mandatory may reduce international comparability, as in most EU member states it is mandatory, and a randomized experiment by Norway (Wilhelmsen, 2012) has shown that the mere fact of the CIS being voluntary or mandatory yielded clearly different results. They found that innovation rates were higher in the voluntary condition of the randomized experiment (all else being equal).
We pre-fill as much as possible, R&D expenditure and personnel figures from the previous year’s R&D survey are shown as a reference. This helps respondents when making estimates for the reference year, but may also perpetuate a mistake made in earlier years.
The fact that countries differ in the practical implementation of CIS may negatively impact comparability between countries. Recommendation of best practices might help here.
Relevance is the degree to which statistics meet current and potential users needs. It includes the production of all needed statistics and the extent to which concepts used (definitions, classifications etc.) reflect user needs. The aim is to describe the extent to which the statistics are useful to, and used by, the broadest array of users. For this purpose, statisticians need to compile information, firstly about their users and their needs.
The CIS is based on a common questionnaire and a common survey methodology, as laid down in the 4th edition of Oslo Manual (2018 edition), in order to achieve comparable, harmonised and high quality results for EU Member States, EFTA countries, Candidates and Associated countries.
12.1. Relevance - User Needs
Besides the needs Eurostat, OECD, and the European Innovation Scoreboard have, very few other user needs are made known to us. Researchers will often add additional questions to the questionnaire, keeping an eye on the questionnaire's length as we have a voluntary CIS.
12.1.1. Needs at national level
| User group | Short description of user group | Main needs for CIS data of the user group Users’ needs |
| 1. Institutions - European Level | Eurostat | |
| 1. Institutions - International organizations | OECD | |
| 1. Institutions - National level | National government | |
| 1. Institutions - Regional level | Regional governments at NUTS 1 level | Various innovation indicators, such as % of innovators by size and industry, amount of R&D expenditure, ... |
12.2. Relevance - User Satisfaction
We do not conduct a user satisfaction survey. Occasionally, we do receive requests for more detailed breakdowns, e.g. for certain sector associations (aggregates of certain NACE codes), for results at NUTS 2 level (provinces) or national level. We handle each of these requests on a case-by-case basis. Generally, we do not provide results at NUTS 2 level (Provinces) but at NUTS 1 level (Regions).
12.3. Completeness
We covered all the compulsory core nace sectors. However, even within the compulsory sectors, some cells are missing. This is because no observations were available for these cells (either because there is no firm in the population, or because none of the surveyed firms answered the questionnaire). Belgium is a small country, therefore we also run into confidentiality issues for certain cells.
We did not ask the following (voluntary) questions: the Strategies and Business environment questions (2.1 - 2.8), the questions on legislation affecting innovation activities (3.14, 3.15) the question on innovations with environmental benefits (3.16, 3.17), the question on the tertiary degree of empoyees (Q4.2), the question on general expenditure (4.6), the question on activities with members of the enterprise group (4.8, 4.9).
12.3.1. Data completeness - rate
Not requested.
13.1. Accuracy - overall
Accuracy in the statistical sense denotes the closeness of computations or estimates to the exact or true values. Statistics are not equal with the true values because of variability (the statistics change from implementation to implementation of the survey due to random effects) and bias (the average of the possible values of the statistics from implementation to implementation is not equal to the true value due to systematic effects).
13.2. Sampling error
That part of the difference between a population value and an estimate thereof, derived from a random sample, which is due to the fact that only a subset of the population is enumerated.
13.2.1. Sampling error - indicators
The main indicator used to measure sampling errors for CIS data is the coefficient of variation (CV).
Coefficient of Variation= (Square root of the estimate of the sampling variance) / (Estimated value)
Formula:
where
13.2.1.1. Coefficient of variations for key variables
Coefficient of variation (%) for key variables by NACE categories and for enterprises with 10 and more employees
| NACE |
Size class |
(1) |
(2) |
(3) |
| Core NACE (B-C-D-E-46-H-J-K-71-72-73) |
Total |
1.16% |
9.67% |
2.28% |
| Core industry (B_C_D_E - excluding construction) |
Total |
1.6% |
14.64% |
3.16% |
| Core Services (46-H-J-K-71-72-73) |
Total |
1.62% |
12.43% |
3.23% |
[1] = Coefficient of variation for the percentage of innovative enterprises (INN) in the total population of enterprises (ENT20)
[2] = Coefficient of variation for the turnover of product innovative enterprises with new or improved products (TUR_PRD_NEW_MKT), as a percentage of total turnover of product innovative enterprises [TUR20,INNO_PRD].
[3] = Coefficient of variation for percentage of product and/or process innovative enterprises (incl. enterprises with abandoned and or on-going activities) involved in any innovation co-operation arrangement [COOP_ALL,INN], as a percentage of innovative enterprises (INN).
13.2.1.2. Variance estimation method
Variances and coefficients of variation were estimated using the default of proc surveymeans in SAS. Our variance estimates took into account our sampling design. However, the fact that imputations were made for missing values was NOT taken into account, nor the fact that nonresponse was a major source of uncertainty in our estimates. Hence, the coefficients of variation reported above can be considered to be lower bounds.
13.3. Non-sampling error
Non-sampling errors occur in all phases of a survey. They add to the sampling errors (if present) and contribute to decreasing overall accuracy. It is important to assess their relative weight in the total error and devote appropriate resources for their control and assessment.
13.3.1. Coverage error
Coverage errors (or frame errors) are due to divergences between the target population and the frame population. The frame population is the set of target population members that has a chance to be selected into the survey sample. It is a listing of all items in the population from which the sample is drawn that contains contact details as well as sufficient information to perform stratification and sampling.
13.3.1.1. Over-coverage - rate
Not requested.
13.3.1.2. Common units - proportion
Not requested.
13.3.1.3. Under covered groups of the target population
There may be some undercoverage of recently founded firms due to the fact that the National Social Security Office Employer database we use as frame population is based on information from the previous year. This is unavoidable, however, given the delay in information available from these firms.
13.3.1.4. Coverage errors in coefficient variation
Variances and coefficients of variation were estimated using the default of proc surveymeans in SAS. Our variance estimates took into account our sampling design. However, the fact that imputations were made for missing values was NOT taken into account, nor the fact that nonresponse was a major source of uncertainty in our estimates. Hence, the coefficients of variation reported above can be considered to be lower bounds.
13.3.2. Measurement error
Measurement errors occur during data collection and generate bias by recording values different than the true ones. The survey questionnaire used for data collection may have led to the recording of wrong values, or there may be respondent or interviewer bias.
13.3.2.1. Measures for reducing measurement errors
We have not collected any systematic evidence regarding measurement error in CIS 2020.
To reduce the risk of measurement errors, we review our questionnaire form every time to improve its clarity and user friendliness. Efforts are made to reduce response burden as much as possible, e.g. by prefilling as many fields as possible. Any comments left on the questionnaire itself or suggestions given by companies are taken into consideration when designing the next CIS. We conduct cognitive interviews when (re)designing our survey forms.
13.3.3. Non response error
Non response occurs when a survey fails to collect data on all survey variables from all the population units designated for data collection in a sample or complete enumeration.
There are two types of non-response:
1) Unit non-response, which occurs when no data (or so little as to be unusable) are collected about a population unit designated for data collection.
a) Un-weighted unit non-response rate (%) = 100*(Number of units with no response or not usable response) / (Total number of in-scope (eligible) units in the sample)
b) Weighted unit non-response rate (%) = 100*(Number of weighted units with no response or not usable response) / (Total number of in-scope (eligible) units in the sample)
2) Item non-response, which occurs when only data on some, but not all survey data items are collected about a population unit designated for data collection.
a) Un-weighted item non-response rate (%) = 100*(Number of units with no response at all for the item) / (Total number of eligible, for the item, units in the sample i.e. filters have to be taken into account)
13.3.3.1. Unit non-response - rate
See below.
13.3.3.1.1. Un-weighted and weighted unit non-response rate by NACE categories and for enterprises with 10 or more employed persons
Un-weighted and weighted unit non-response rate by NACE categories and for enterprises with 10 or more employees
| NACE | Number of eligible units with no response | Total number of eligible units in the sample | Un-weighted unit non-response rate (%) | Weighted unit non-response rate (%) |
| Core NACE (B-C-D-E-46-H-J-K-71-72-73) | 3190 | 8241 | 39% | 40% |
| Core industry (B_C_D_E - excluding construction) | 1299 | 3501 | 37% | 39% |
| Core Services (46-H-J-K-71-72-73) | 1891 | 4740 | 40% | 40% |
The number of eligible units is the number of sample units, which indeed belong to the target population.
13.3.3.1.2. Maximum number of recalls/reminders before coding
Two written reminders are sent out by post, phone call and e-mail reminders are sent to a limited number of enterprises. (Enterprises in Flanders for whom e-mail addresses were available were sent three e-mail reminders and one paper reminder. Enterprises for whom no e-mail address was available (a small minority) were sent two paper reminders. Phone calls were made during three months to encourage enterprises to respond.)
13.3.3.2. Item non-response - rate
See below.
13.3.3.2.1. Item non-response rate for Turnover (in Core NACE: B-C-D-E-46-H-J-K-71-72-73 enterprises with 10 or more employed persons)
Item non-response rate for Turnover (in Core NACE: B-C-D-E-46-H-J-K-71-72-73 enterprises with 10 or more employees).
| Item non-response rate (un-weighted) | Imputation | If imputed, describe method used, mentioning which auxiliary information or stratification is used | |
| Turnover | 11% | Y | Administrative data are available for turnover. As EBS recommends us to use administrative data whenever feasible (to reduce response burden), we treat an administrative value for turnover as if it were a response. Only when there was no response given for turnover NOR is there any administrative value, we consider this to be nonresponse. Imputations are ratio means, for nace x size cells. We use the SAS routine that was originally made available by Eurostat for CIS 4 and that we have updated for use for CIS 2020. |
13.3.3.2.2. Item non response rate for new questions
Item non-response rate for new questions in CIS t (in Core NACE: B-C-D-E-46-H-J-K-71-72-73 enterprises with 10 or more employees)
| NEW QUESTIONS IN CIS 2020 | Inclusion in national questionnaire | Item non response rate (un-weighted) | Comments |
| 2.2 Market conditions faced by enterprise | Not applicable: question was not included | ||
| 2.8 Factors related to climate change | Not applicable: question was not included | ||
| 3.16 Innovations with environmental benefits | Not applicable: question was not included | ||
| 3.17 Factors driving environmental innovations | Not applicable: question was not included |
13.3.4. Processing error
We are not aware of any processing errors
13.3.5. Model assumption error
Not requested.
Timeliness and punctuality refer to time and dates, but in a different manner.
14.1. Timeliness
The timeliness of statistics reflects the length of time between data availability and the event or phenomenon they describe.
14.1.1. Time lag - first result
Timeliness of national data – date of first release of national level : July 1st, 2022 (delivery to Eurostat)
14.1.2. Time lag - final result
Not requested.
14.2. Punctuality
Punctuality refers to the time lag between the release date of data and the target date on which they were scheduled for release as announced officially.
14.2.1. Punctuality - delivery and publication
Date of transmission of complete and validated data to Eurostat (Number of days between that data and 30 June 2022) : 3
Comparability aims at measuring the impact of differences in applied statistical concepts and definitions on the comparison of statistics between geographical areas, non-geographical domains, or over time.
The coherence of statistical outputs refers to the degree to which the statistical processes by which they were generated used the same concepts (classifications, definitions, and target populations) and harmonised methods. Coherent statistical outputs have the potential to be validly combined and used jointly.
15.1. Comparability - geographical
We use the same statistical concepts and definitions in all regions, and as far as we're aware, those are the same concepts and definitions used in the rest of the EU.
15.1.1. Asymmetry for mirror flow statistics - coefficient
Not requested.
15.1.2. National questionnaire – compliance with Eurostat model questionnaire
Methodological deviations from the CIS Harmonised Data Collection (HDC)
| Questions not included in national questionnaire compared to HDC | Comment |
| Q. 2.1 - 2.8, Q 3.14, Q 3.15, Q 3.16, Q 3.17, Q 4.2, Q 4.6, Q 4.8, and Q 4.9 | Since our survey is voluntary, we need to limit the number of questions included in our survey form. |
| Changes in the filtering compared to HDC | Comment |
| Yes | Both innovators and non-innovators had to respond to the question on expenditure for innovation and R&D (Q 3.8). All enterprises indicating any kind of cooperation were asked to give more details on where and who their cooperation partners were (Q3.13) even if this cooperation did not occur in context of R&D or innovation. In reporting our results to Eurostat, we omitted those cases that only had cooperation outside of an R&D or innovation context. |
15.1.3. National questionnaire – additional questions
Methodological deviations from the CIS Harmonised Data Collection (HDC)
| Additional questions in national questionnaire (not included in HDC) | Comment |
| We also asked for numbers for R&D personnel in 2020: both head counts and FTE, and we asked this separately for internal R&D personnel and for external R&D personnel, as well as a more detailed breakdown of costs incurred for R&d in 2020. | |
| We asked for new-to-world product innovations. | |
| Did your enterprise conduct R&D in biotechnology, biochemistry, nanotechnology, AI? | |
| Did the process innovations lead to cost reductions? | |
| We asked for the impact of the COVID 19 crisis on innovation activities |
15.2. Comparability - over time
Due to important methodological changes driven by Oslo Manual 2018, CIS 2018 and CIS 2020 cannot be directly compared with previous CIS waves.
15.2.1. Length of comparable time series
Not requested.
15.3. Coherence - cross domain
See the comparison between SBS and CIS data in the section 15.3.3 below.
15.3.1. Coherence - sub annual and annual statistics
Not requested.
15.3.2. Coherence - National Accounts
Not requested.
15.3.3. Coherence – Structural Business Statistics (SBS)
This part compares key variables for aggregated CIS data with SBS data
Definition of relative difference between CIS and SBS data: DIFF = (SBS/CIS)*100
Comparison between SBS and CIS data (relative difference) by NACE categories and for enterprises with 10 or more employees
| NACE | Size class | Number of enterprises (SBS/CIS)* | Number of employees (SBS/CIS)* | Total Turnover (SBS/CIS)* |
| Core NACE (B-C-D-E-46-H-J-K-71-72-73) | Total | We do not have access to SBS data | ||
| Core industry (B_C_D_E - excluding construction) | Total | We do not have access to SBS data | ||
| Core Services (46-H-J-K-71-72-73) | Total | We do not have access to SBS data |
* Numbers are to be provided for the last year of the reference period (t)
15.4. Coherence - internal
Not requested.
Confidential information on the production cost of the CIS.
16.1. Cost
The assessment of costs associated with a statistical product is a rather complicated task since there must exist a mechanism for appointing portions of shared costs (for instance the business register or shared IT resources and dissemination channels) and overheads (office space, utility bills etc). The assessment must become detailed and clear enough so that international comparisons among agencies of different structures are feasible.
For measuring the cost on statistical offices, Eurostat proposes to make use of the following very short calculation, even if Eurostat is aware of the fact that such a measure may be complicated.
| Costs for the statistical authority | In thousands of national currency |
| Total cost (in thousand currency units) | 286 |
| Staff cost (in %) | 68.7% |
| Other direct cost (in %) | 47 |
| Overheads (in %) | 44 |
National currency used for this amount:
17.1. Data revision - policy
Not requested.
17.2. Data revision - practice
Not requested.
17.2.1. Data revision - average size
Not requested.
18.1. Source data
See below
18.1.1. Sampling frame (or census frame)
Due to confidentiality constraints the official Belgian business register could not be used. Instead, we used as frame population the register available from the Belgian National Social Security Office which contains all active employers in Belgium. We used its October 2020 version. This register was agreed upon by Statistics Belgium as being statistically equivalent to the official business register. The total number of firms in the population was 14,977.
18.1.2. Sampling design
Belgium is composed of 3 Regions (at NUTS 1 level): Flanders, Brussels, and Wallonia. Each Region is endowed with its own statistical office. Therefore, three separate
samples were drawn.
1. For the Brussels Region
Strata considered were formed by crossing size and NACE division (at 2-digit level). A census was performed for all large and medium-large enterprises, except for a sampling of the NACE 46 (sampling rate around 61%), Nace 49 (sampling rate around 83%), Nace 59 (sampling rate around 65%), Nace 62 (sampling rate around 64%), Nace 64 (sampling rate around 89%), Nace 66 (sampling rate around 84%), Nace 71 (sampling rate around 70%), Nace 73 (sampling rate around 84%).
2. For the Walloon Region
Two dimensions were used for the stratification structure of the sampling: size and NACE sector. Census was done for all large firms, for medium-sized firms belonging to NACE 8-46 and 58-73, as well as small firms belonging to NACE 20-21, 26-27, and 72.
For the remaining medium-sized (size=2) and small firms (size=1), the following sampling rates apply:
• Nace8-9, 13-15, 31-32, 35-39 about 68% (size 1)
• Nace 10-12 about 57% (size 1)
• Nace 16-18 about 81% (size 1)
• Nace 22-23, 33 about 66% (size 1)
• Nace 46 about 38% (size 1)
• Nace 49-53 about 47% (size 1)
• Nace 58, 64-66, 73 about 73% (size 1)
• Nace 24-25 about 55% (size 1)
• Nace 59-63, 71 about 43% (size 1)
• Nace 28-30 about 75% (size 1)
• Nace 49-53 about 79% (size 2)
3. For the Flemish Region
Besides firm size and sector, other stratification variables that were taken into account for sampling in the Flanders region were whether or not a firm was known to have continuous R&D spending, whether or not a firm was active in biotechnology, nanotechnology or artificial intelligence and whether or not a firm recently received public funding for R&D or was a university spinoff. The inventory of firms with continuous R&D spending as obtained from the 2020 R&D survey was used as a base for the first of these variables.
Census sampling was done for all large size firms (250 or more employees), for all medium size firms
(50-249 employees) and for small size firms (10-49 employees) of NACE 19-22, 26-30, 59-63, and 71-72. Census sampling was also done of the small size firms known to have continuous R&D spending in the other core NACE sectors, for all small size firms active in biotechnology, nanotechnology or artificial intelligence and for all small size firms that recently received public funding for R&D or were university spinoffs.
For the remaining small size firms first sampling rates were set that would meet the Eurostat precision criteria for NACE sectors grouped according to their technology level: low-tech industry (NACE 5-18, 31-39), medium low-tech industry (NACE 19, 22-25, 33) and low-tech services (NACE 46,49-53, 58, 64-66 and 73). Neymann allocation was then used for more fine-grained NACE aggregates within those technology level groupings. Taking into account expected levels of non-response, a minimum sample size of 40 firms was set for each NACE aggregate.
18.1.3. Target population and sample size
| Sample/census indicator | Number of enterprises |
| Target population | 14977 |
| Sample | 8241 |
| In case of combination sample/census: | |
| Sampled units | 2757 |
| Enumerated units/census | 5484 |
| Overall sample rate (overall sample/target population) | 55.02% |
18.1.4. Data source for pre-filled variables
Variables and indicators filled or prefilled from other sources. Bel-first (commercial database containing publicly available balance sheet data in database format)
| Variables/Indicators | Source | Reference year |
| Turnover | Belfirst | 2018 |
| Number of employees | Belfirst | 2018 |
| Member of an enterprise group | Belfirst | 2020 |
18.1.5. Data source and variables used for derivation and weighting
| Item | Response |
| Data source used for deriving population totals | No calibration was used, population totals are simply derived from the survey data themselves, and weights are based on number of firms |
| Variables used for weighting | Weights are simply N/n, the number of firms in the population over the number of firms in the realized sample. |
18.2. Frequency of data collection
According to the Commission Regulation (UE) 995/2012, the innovation statistics shall be provided to Eurostat every two years in each even year t+18.
18.3. Data collection
See below
18.3.1. Survey participation
The survey is voluntary.
18.3.2. Survey type
We used a combination of sample and census, depending on the size of the population in the various strata, so as to make sure to match Eurostats quality criteria.
18.3.3. Combination of sample survey and census data
For the Brussels Region
All large and medium-sized enterprises were included, random sampling was done for NACE 46, 49, 59, 62, 64, 66, 71, and 73 for all remaining small enterprises.
For the Walloon Region
- Among large enterprises: a census was performed (250 employees or more).
- Among medium-sized firms (50-249 employees), random sampling was performed on the following 2-digit NACE sector: 49-53
- Among small firms (10-49 employees), random sampling was performed on each of the following 2-digit NACE sector:
- 8-9, 13-15, 31-32, and 35-39
- 10-12
- 16-18
- 22-23, and 33
- 46
- 49-53
- 58, 64-66, and 73
- 24-25
- 59-63, and 71
- 28-30
For the Flemish Region
Besides firm size and sector, other stratification variables that were taken into account for sampling in the Flanders region were whether or not a firm was known to have continuous R&D spending, whether or not a firm was active in biotechnology, nanotechnology or artificial intelligence and whether or not a firm recently received public funding for R&D or was a university spinoff. The inventory of firms with continuous R&D spending as obtained from the 2020 R&D survey was used as a base for the first of these variables.
Census sampling was done for all large size firms (250 or more employees), for all medium size firms
(50-249 employees) and for small size firms (10-49 employees) of NACE 19-22, 26-30, 59-63, and 71-72. Census sampling was also done of the small size firms known to have continuous R&D spending in the other core NACE sectors, for all small size firms active in biotechnology, nanotechnology or artificial intelligence and for all small size firms that recently received public funding for R&D or were university spinoffs.
For the remaining small size firms first sampling rates were set that would meet the Eurostat precision criteria for NACE sectors grouped according to their technology level: low-tech industry (NACE 5-18, 31-39), medium low-tech industry (NACE 19, 22-25, 33) and low-tech services (NACE 46,49-53, 58, 64-66 and 73). Neymann allocation was then used for more fine-grained NACE aggregates within those technology level groupings. Taking into account expected levels of non-response, a minimum sample size of 40 firms was set for each NACE aggregate.
18.3.4. Census criteria
Only in the more populated cells in our sampling design, random sampling was applied. Generally, these were the smaller, and more low-tech firms.
1. For the Brussels Region
Census was done for all large and medium-sized firms, as well as small firms belonging to NACE 20-22, 33, 26- 30, 59, 61-63, and 71-72.
2. For the Walloon Region
Census was done for all large firms, for medium-sized firms belonging to NACE 8-46 and 58-73, as well as small firms belonging to NACE 20-21, 26-27, and 72.
3. For the Flemish Region
Besides firm size and sector, other stratification variables that were taken into account for sampling in the Flanders region were whether or not a firm was known to have continuous R&D spending, whether or not a firm was active in biotechnology, nanotechnology or artificial intelligence and whether or not a firm recently received public funding for R&D or was a university spinoff. The inventory of firms with continuous R&D spending as obtained from the 2020 R&D survey was used as a base for the first of these variables.
Census sampling was done for all large size firms (250 or more employees), for all medium size firms
(50-249 employees) and for small size firms (10-49 employees) of NACE 19-22, 26-30, 59-63, and 71-72. Census sampling was also done of the small size firms known to have continuous R&D spending in the other core NACE sectors, for all small size firms active in biotechnology, nanotechnology or artificial intelligence and for all small size firms that recently received public funding for R&D or were university spinoffs.
For the remaining small size firms first sampling rates were set that would meet the Eurostat precision criteria for NACE sectors grouped according to their technology level: low-tech industry (NACE 5-18, 31-39), medium low-tech industry (NACE 19, 22-25, 33) and low-tech services (NACE 46,49-53, 58, 64-66 and 73). Neymann allocation was then used for more fine-grained NACE aggregates within those technology level groupings. Taking into account expected levels of non-response, a minimum sample size of 40 firms was set for each NACE aggregate.
18.3.5. Data collection method
Data collection method
| Survey method | Yes/No | Comment |
| Face-to-face interview | No | |
| Telephone interview | Yes | A limited number of responses resulted from telephone interviews: some respondents asked to be interviewed over the phone, others who initially refused to respond when contacted over the phone to remind them of the survey, were converted to respondents |
| Postal questionnaire | Yes | |
| Electronic questionnaire (format Word or PDF to send back by email) | Yes | |
| Web survey (online survey available on the platform via URL) | Yes | |
| Other | No |
18.4. Data validation
Not requested.
18.5. Data compilation
Operations performed on data to derive new information according to a given set of rules.
18.5.1. Imputation - rate
Imputation is the method of creating plausible (but artificial) substitute values for all those missing.
Definition of imputation rate:
Imputation rate (for the variable x) (%) = 100*(Number of replaced values) / (Total number of values for a given variable)
Definition of weighted imputation rate:
Weighted imputation rate= 100*(Number of total weighted replaced values) / (Total number of weighted values for a given variable)
18.5.1.1. Imputation rate for metric variables
Imputation rate for metric variables by NACE categories and for enterprises with 10 or more employees:
| NACE | Size class | Total Turnover (1) | Turnover from products new to the market (2) | R&D expenditure in-house (3) | |||
| Unweighted | Weighted | Unweighted | Weighted | Unweighted | Weighted | ||
| Core NACE (B-C-D-E-46-H-J-K-71-72-73) | Total | 10% | 14% | 10% | 11% | 7% | 8% |
| Core industry (B_C_D_E - excluding construction) | Total | 9% | 14% | 10% | 12% | 8% | 9% |
| Core Services (46-H-J-K-71-72-73) | Total | 11% | 15% | 11% | 10% | 7% | 7% |
(1) = Total turnover in the last year of the reference period (t) (TUR)
(2) = Share of the turnover in the last year of the reference period (t) due to new or improved product new to the market in the total turnover for product innovative enterprises TUR_PRD_NEW_MKT/TUR(INNO_PRD)
(3) = R&D expenditure performed in-house (EXP_INNO_RND_IH)
18.5.2. Weights calculation
Weights calculation method for sample surveys
| Method | Selected applied method | Comments |
| Inverse sampling fraction | Weights are simply the inverse of the realized sampling fractions. | |
| Non-respondent adjustments | None made | |
| Other | A limited number of specific observations are considered separately by giving them a weight=1. These are observations for which the R&D expenditures are so high compared to the rest of the sample that multiplying these observations by more than one would bias the final result for R&D and innovation expenditures variables. |
18.6. Adjustment
Not applicable
18.6.1. Seasonal adjustment
Not requested.
The Community Innovation Survey (CIS) is a survey about innovation activities in enterprises. The survey is designed to collect information on types of innovation, processes of development of innovation like cooperation patterns, financing and expenditure, objectives of innovation activities or barriers for initiating or implementing innovation.
The CIS provides statistics by type of innovators, economic activity, and size class of enterprises. The survey is currently carried out every two years across the EU Member States, EFTA countries and EU candidate countries.
In order to ensure comparability across countries, Eurostat together with the member countries develops a Harmonised Data Collection (HDC) questionnaire and drafts the methodological recommendations for implementation of each survey round.
CIS 2020 is a second in a row to implement concepts and methodology of the Oslo Manual 4th Edition revised in 2018. Please refer to the Annex section of the European metadata (ESMS) for details of the time coverage of collected indicators. Community Innovation Survey – new features.
The legal framework for CIS since 2012 is the Commission Regulation No 995/2012 that establishes the quality conditions for the data collection and transmission and identifies the obligatory cross-coverage of economic sectors, size class of enterprises and innovation indicators. The target population are enterprises with at least 10 employees classified in the core NACE economic sectors (see 3.3). Further activities may be covered on a voluntary basis in national datasets. Most statistics are based on the 3-year reference period (t, t-1, t-2), but some use only one calendar year (t or t-2). Please refer to the Annex section of the European metadata (ESMS) for details of the time coverage of collected indicators.
27 May 2024
The description of concepts, definitions and main statistical variables is available in CIS 2020 European metadata file (ESMS) Results of the community innovation survey 2020 (CIS2020) (inn_cis12) in Eurostat database.
The legal unit was used as the statistical unit.
Core target population are all enterprises in CORE NACE activities (see 3.3.1) with 10 or more employees.
Belgium is composed of 3 regions (at NUTS 1 level): Brussels, Flanders, and Wallonia. Each region is endowed with its own statistical office. Thus, three separate samples were drawn.
For CIS 2020, the time covered by the survey is the 3-year period from the beginning of 2018 to the end of 2020.
Some questions and indicators refer to one year — 2020.
The list of indicators covering the 3-year period and referring to one year according to the HDC is available in the Annex section of the European metadata (ESMS).
Accuracy in the statistical sense denotes the closeness of computations or estimates to the exact or true values. Statistics are not equal with the true values because of variability (the statistics change from implementation to implementation of the survey due to random effects) and bias (the average of the possible values of the statistics from implementation to implementation is not equal to the true value due to systematic effects).
CIS indicators are available according to 3 units of measure:
NR: Number for number of enterprises and number of persons employed.
THS_EUR: Thousands of euros. All financial variables are provided in thousands of euros, i.e. Turnover or Innovation expenditure.
PC: Percentage. The percentage is the ratio between the selected combinations of indicators.
Operations performed on data to derive new information according to a given set of rules.
See below
CIS is conducted and disseminated at a two-year interval in even years.
The timeliness of statistics reflects the length of time between data availability and the event or phenomenon they describe.
We use the same statistical concepts and definitions in all regions, and as far as we're aware, those are the same concepts and definitions used in the rest of the EU.
Due to important methodological changes driven by Oslo Manual 2018, CIS 2018 and CIS 2020 cannot be directly compared with previous CIS waves.
