Research and development (R&D) (rd)

National Reference Metadata in Single Integrated Metadata Structure (SIMS)

Compiling agency: National Documentation Centre (EKT)


Eurostat metadata
Reference metadata
1. Contact
2. Metadata update
3. Statistical presentation
4. Unit of measure
5. Reference Period
6. Institutional Mandate
7. Confidentiality
8. Release policy
9. Frequency of dissemination
10. Accessibility and clarity
11. Quality management
12. Relevance
13. Accuracy
14. Timeliness and punctuality
15. Coherence and comparability
16. Cost and Burden
17. Data revision
18. Statistical processing
19. Comment
Related Metadata
Annexes (including footnotes)



For any question on data and metadata, please contact: Eurostat user support

Download


1. Contact Top
1.1. Contact organisation

National Documentation Centre (EKT)

1.2. Contact organisation unit

RDI Metrics and Services Department

1.5. Contact mail address

56, Zefyrou, GR-17564, P. Faliro


2. Metadata update Top
2.1. Metadata last certified 17/11/2023
2.2. Metadata last posted 17/11/2023
2.3. Metadata last update 17/11/2023


3. Statistical presentation Top
3.1. Data description

Statistics on Government R&D (GOVERD) measure research and experimental development (R&D) performed in the Government sector, i.e. R&D expenditure and R&D personnel. In line with this objective the target population for the national R&D survey of the Government sector should consist of all R&D performing units (including all R&D performers – occasional and continuous, known and unknown - in all branches and size classes) belonging to this sector.

Main concepts and definitions used for the production of R&D statistics are given by the OECD (2015), Frascati Manual 2015: Guidelines for Collecting and Reporting Data on Research and Experimental Development, The Measurement of Scientific, Technological and Innovation Activities, which is the internationally recognised standard methodology for collecting R&D statistics and by the European Business Statistics Methodological Manual on R&D Statistics (EBS Methodological Manual on R&D Statistics). 

Since the beginning of 2021, the collection of R&D statistics is based on Commission Implementing Regulation (EU) No 2020/1197 of 30 July 2020. The Regulation sets the framework for the collection of R&D statistics and specifies the main variables of interest and their breakdowns at predefined level of detail. Statistics on science, technology and innovation were collected until the end of 2020 based on the Commission Implementing Regulation (EU) No 2012/995 concerning the production and development of Community statistics on science and technology. 

3.2. Classification system
3.2.1. Additional classifications
Additional classification used Description
   
   
   
3.3. Coverage - sector

See below.

3.3.1. General coverage
Definition of R&D

The definition of R&D statistics is aligned with the FM 2015 definition

Fields of Research and Development (FORD)

No deviation from FM 2015. Both personnel(researchers and total) data and expenditure data are collected and classified according to FORD classification (6 major fields), covering both NSE and SSH.

Socioeconomic objective (SEO by NABS)

Since 2012 (reference year 2011 onwards) data is available.

3.3.2. Sector institutional coverage
Government sector  

Coverage of GOV is in line with the FM
recommendations and ESA 13 classification.
It covers public institutions administered or/and
financed by Ministries (public research centers,
archaeological and cultural institutions, public
hospitals, agricultural institutions, etc.), as well as
Greek Public Independent Authorities.

Hospitals and clinics  

University hospitals are included in the higher
education sector (HES). Private and PNP hospitals are
included in the BES and PNP sector respectively.
Government hospitals and Military hospitals are
included in the GOV sector since 2012
(for reference year 2011 and onwards).

Inclusion of units that primarily do not belong to GOV  Not applicable.
3.3.3. R&D variable coverage
R&D administration and other support activities  Included, according to FM §2.1.2.2.
External R&D personnel

Since reference year 2011, the reported total R&D personnel included both internal and external personnel, as required by the FM 2005 and FM 2015. However, no detailed data are available for each category (as well as relevant breakdowns, such as gender).

In the frame of the 2020_EL_RDI project, EKT has applied five distinct R&D personnel categories based on the most common types of employment in Greek R&D institutional units. Data are separately collected for each category, allowing for an accurate measurement of both the internal and the external personnel, of the R&D performing unit, across various breakdowns (gender, education level, age, etc.).

The R&D GOV questionnaire provides an introductory section (Section B: Personnel engaged in R&D activities), which guides the respondents to identify the human resources that contribute to their R&D activities according to their contractual relationship (i.e., employment status) with the enterprise. Detailed guidelines and examples, have been developed and provided to enterprises to assist them in computing the distribution of their personnel into the different categories.

More specifically, respondents are asked to indicate whether they employ individuals (Researchers and/or Other R&D personnel) under the following five employment status categories:

B1.1. Internal permanent personnel engaged in R&D activities:
B1.2 Internal temporary personnel engaged in R&D activities:
B1.3 External contributors engaged in R&D activities:
B1.4. External personnel of Greek HEIs engaged in R&D activities:
B1.5. Other external personnel engaged in R&D activities:

Clinical trials  Clinical trials (phases 1, 2, 3 and occasionally 4) undertaken by pharmaceutical companies are covered.
3.3.4. International R&D transactions
Receipts from rest of the world by sector - availability  

Funding from abroad is covered by all sectors and is broken down into: European Commission
(e.g. Framework 7, HORIZON 2020 and other EU Programs), Foreign Business enterprises, International
Organizations, Other sources.

Payments to rest of the world by sector - availability

Since the 2013 survey, an additional section on extramural expenditure has been added to GOV questionnaire, including information on extramural expenditure to abroad (enterprises and other organizations separately). However, it should be noted that data are collected only from R&D Performers and do not cover the total national payments to abroad.

3.3.5. Extramural R&D expenditures

According to the Frascati Manual, expenditure on extramural R&D (i.e. R&D performed outside the statistical unit) is not included in intramural R&D performance totals (FM, §4.12).

Data collection  on extramural R&D expenditure (Yes/No) Y
Method for separating extramural R&D expenditure from intramural R&D expenditure Since the 2013 survey, an additional section on extramural expenditure has been added to the questionnaire, including information on extramural expenditure to abroad (enterprises and other organisations separately). 
Difficulties to distinguish intramural from extramural R&D expenditure No particular difficulties reported.
3.4. Statistical concepts and definitions

See below.

3.4.1. R&D expenditure
Coverage of years  Calendar year.
Source of funds  No divergence from the Frascati Manual recommendations; additional breakdown are foreseen to collect information for national purposes (e.g. ESPA 2014-2020) and to help respondents to accurately identify the various sources of R&D funds. 

Since 2021, following the realization of the 2020_EL_RDI project and Eurostat and STI WGs guidelines, two additional breakdowns for R&D funding have been added: internal /external and transfer/ exchange.  

The information required is collected from respondents. More specifically, the questionnaire covers the following sources of funds while respondents are asked to define the transfer/exchange component for each category:

Government:

  • Ordinary budget
  • Public Investment Budget - excluding ESPA 2014-2020
  • Other (Regions, Municipalities, Special Funds, etc.)
  • Partnership Agreement ESPA 2014-2020
  • Own funds

Business enterprises sector:

  • Private Greek enterprises
  • Public enterprises

Higher Education Sector

Private non-profit institutions

Rest of the World:

  • European Commission (e.g. Horizon Europe, Horizon 2020, other EU programmes)
  • Foreign enterprises
  • International Organisations (United Nations, etc.)
  • Other foreign organisations

 

Type of R&D  

Since 2012 (reference year 2011 onwards)
breakdowns regarding type of R&D are available
for the total intramural R&D expenditure.

Type of costs

Type of costs Since reference year 2011 onwards, the available breakdowns regarding type of costs included:

Current expenditure:

  • Current - labour cost
  • Current - other expenditure

Capital expenditure:

  • Capital - land and buildings
  • Capital - instruments and equipment.

Further breakdowns for type of costs according to Eurostat’s guidelines, as defined in the new (Version 4) R&D Data Structure Definition (DSD). The following list presents the categories by type of cost:

Current costs:

  • Labour costs for internal R&D personnel
  • Other current costs: - Other current costs / External R&D personnel - Other current costs / Other costs

Capital costs:

  • Land and buildings
  • Machinery and equipment
  • Capitalized computer software
  • Other intellectual property products
Defence R&D - method for obtaining data on R&D expenditure Information about defence R&D is separately available for the GOV sector only (NABS classification).
3.4.2. R&D personnel

See below.

3.4.2.1. R&D personnel – Head Counts (HC)
Coverage of years   Calendar year.
Function  

Information about the two occupational categories
(researchers and other R&D personnel) is
collected. Since 2012 (reference period 2011
onwards) information about all occupations is also
available by sex, educational level (and sex),
by major field of science (and sex) and by region
(and sex).

Qualification  

Qualification is available for all occupational
categories and sex. Data are separately available
for ISCED 2011 level 8, ISCED 2011 levels 5, 6
and 7 and ‘other qualification’ in line with the new
classification that has been introduced with the
Com. Reg. 995/2012

Age  

Since 2012 (reference year 2011 onwards)
information about age (and sex) of the researchers
has been collected in head counts.

Citizenship  

Since 2012 (reference year 2011 onwards)
information about citizenship of the researchers
has been collected in head counts.

3.4.2.2. R&D personnel – Full Time Equivalent (FTE)
Coverage of years  Calendar year.
Function  

Information about the two occupational categories
(researchers and other R&D personnel) is
collected. Since 2012 (reference period 2011
onwards) information about all occupations is also
available by sex, educational level (and sex),
by major field of science (and sex) and by region
(and sex).

Qualification  

Qualification is available all qualification
categories and sex. Data are separately available
for ISCED 2011 level 8, ISCED 2011 levels 5, 6
and 7 and ‘other qualification’ in line with the new
classification that has been introduced with the
Com. Reg. 995/2012.

Age  Not available / not collected.
Citizenship  Not available / not collected.
3.4.2.3. FTE calculation

Reporting units made the calculation of FTEs following the questionnaire guidelines that have been drafted in line with FM recommendations (§ 333). Information about how calculations were performed has been provided by respondents in the metadata chapter of the questionnaire. Note that since 2017, FTEs less than 10% are not reported as R&D activities.

The majority of the institutions in the government sector have reported that they calculated FTE using the internal time-sheets kept by the institutions or by applying coefficients as reported by directors / heads of the institutions. Combination of the two has also been reported. To a lesser extent, institutions have also reported the use of other methods, such as the application of different coefficients on occupation or type of contract, the implementation of survey to the staff, etc. R&D coefficients, derived from a time use survey realized by EKT in 2015, were applied for the calculation of the R&D share (and FTEs) in the Public Hospitals.

 

3.4.2.4. R&D personnel - Cross-classification by function and qualification
Cross-classification Unit Frequency
Since 2012 (reference year 2011 onwards), headcount (HC) and Full Time Equivalent  (FTE) data for R&D personnel are cross-classified by occupation and qualification (and sex) for all R&D sectors.  HC and  FTEs  Annual for occupation (and sex) and qualification.
     
     
3.5. Statistical unit

Government institutions as listed in the statistical Register of General Government Entities (S13) that is maintained by ELSTAT (Hellenic Statistical Authority). Merges/ abolitions of institutions are regularly checked and statistical units are modified accordingly.

3.6. Statistical population

See below.

3.6.1. National target population

The target population is the population for which inferences are made. The frame (or frames, as sometimes several frames are used) is a device that permits access to population units. The frame population is the set of population units which can be accessed through the frame and the survey data really refer to this population of institutional units.

The objective of the European R&D statistics is to cover all intramural R&D activities. In line with this objective, the target population for the national R&D survey of the Government Sector should consist of all R&D performing units (including known R&D performers or assumed to perform R&D). In practise however, countries in their R&D surveys might have difficulty in identifying R&D activities at the municipality level. 

 

  Target population when sample/census survey is used for collection of raw data Target population when administrative data or pre-compiled statistics are used
Definition of the national target population  

The target population is based on the GOV R&D Directory which is developed by EKT and includes those institutions of the general government (ESA S13) that are known to regularly perform R&D (core institutions) or to occasionally perform R&D (perimeter institutions), with the exception of higher education institutions and public enterprises. The Directory is regularly updated. It is also checked against the statistical Register of General Government Entities that is maintained by ELSTAT (the Hellenic Statistical Authority) which is updated annually to reflect any recent changes in the perimeter of the Greek General Government (mergers/ abolitions, e.tc.). Both core and perimeter institutions are covered by a census survey.

 

Administrative data are
collected as supplementary data for a part of the target
population of the census survey,
namely public hospitals,
archaeological and cultural institutions. These data are used
to provide direct government
funding information and to
cross-check data collected from
the census surveys.

Estimation of the target population size  

Regular (core) and occasional
(perimeter) R&D performers: 336

 

Regular (core) and occasional
(perimeter) R&D performers: 336

3.6.2. Frame population – Description

In ESS countries, the frame population for GOV R&D statistics is defined as the list of all the institutional units classified by the national accounts (ESA) as included in the General government (S.13), with the exclusion of those units included in the Higher education sector (HES).

 

Method used to define the frame population  

The survey frame comprises institutions included
in the statistical Register of General Government
Entities (S13) that is maintained by ELSTAT
(Hellenic Statistical Authority) with the exclusion
of those units belonging to the Higher Education
Sector. A few public enterprises listed in S13 are
also excluded.

Methods and data sources used for identifying a unit as known or supposed R&D performer

The target population is based on the GOV R&D Directory at EKT and includes those institutions of the general government (ESA S13) that are known to perform or are very likely to perform R&D, with the exception of higher education institutions and public enterprises. To this end, the R&D survey frame for GOV sector is the GOV R&D directory (which is a subset of the statistical Register of General Government Entities (S13)) and includes government research centers (100% R&D performers) as well as other government institutions that perform R&D along with other activities, on regular or occasional basis.

The development of GOV R&D Directory started with a screening exercise which was conducted on all GOV potential R&D performers during the first year of the implementation of the survey by EKT (in 2012-2013 with reference to 2011 data).

During the second round (reference year 2013), the directory has been updated based on the answers from the first round, the institutions organizational changes (e.g. merges or splits of institutions), the inclusion of R&D performers not previously known, that have been identified as new from the GBARD survey or from information retrieved from administrative sources (e.g. Monitoring Information System (M.I.S.) of “NSRF 2014-2020”, eCORDA database containing information about R&D projects financed by EU research programs (Horizon 2020, FP7).

Below is a description of the GOV R&D Directory at EKT:

A) All Research Centershat are supervised by the General Secretariat for Research and Innovation (GSRI) (Resarch centers in alphabetic order in Greek): the National Observatory of Athens, the National Hellenic Research Foundation, The Centre for Research and Technology (including the Center for Research and Technology, Thessaly), the National Center for Scientific Research DEMOKRITOS, the Hellenic Centre for Marine Research, the National Centre for Social Research, the Biomedical Research Foundation Academy of Athens the Hellenic Pasteur Institute, the Alexander Fleming Biomedical Sciences Research Center, the Athena-Research and Innovation Center in Information, Communication and Knowledge Technologies, the Foundation for Research & Technology Hellas. Moreover, the technological bodies that are supervised by the GSRI: the Greek Atomic Energy Commission the Patras Science Park and
the Thessaloniki Science Center & Technology Museum NOESIS).

B) Other Public Research Institutions supervised by different Ministries
(indicative and non-exhaustive list of GOV institutions is the following:the Academy of Athens, , the Hellenic Agricultural Organization DEMETRA, the Benaki Phytopathological Institute, the Center for Renewable Energy Sources and Saving, the Mediterranean Agronomic Institute of Chania, the Computer Technology Institute and Press Diophantus, etc.).

C) Entities supervised by the Ministry of Culture: Ephorates of Antiquities, Ephorate of Palaeoanthropology and Speleology, Ephorate of Underwater Antiquities, Management of the National Archive of Monuments, Acropolis Restoration Service, archaeological sites and museums, Cultural heritage organizations, etc.

D) Public hospitals

E) Public independent authorities, other government organizations, etc.
The Directory is regularly updated against the statistical Register of
General Government Entities (S13) that is maintained by ELSTAT
(for any mergers/abolitions). It is also enriched with government
institutions belonging to S13 list, in cases of a clear evidence for the
performance of R&D activities (information from GBARD surveys,
eCORDA, NSRF national development programs etc.).

Inclusion of units that primarily do not belong to the frame population  No
Systematic exclusion of units from the process of updating the target population  No
Estimation of the frame population  

Frame population was the same as the target population, i.e. 336
institutions for 2021.

3.7. Reference area

Not requested.

3.8. Coverage - Time

Not requested. 

3.9. Base period

Not requested. 


4. Unit of measure Top

For personnel: HC, FTE

For costs and funding: MIO_NAC (Millions of National Currency)


5. Reference Period Top

Reference year: 2021.        


6. Institutional Mandate Top
6.1. Institutional Mandate - legal acts and other agreements

See below.

6.1.1. European legislation
Legal acts / agreements

Since the beginning of 2021, the collection of R&D statistics is based on Commission Implementing Regulation (EU) No 2020/1197 of 30 July 2020 laying down technical specifications and arrangements pursuant to Regulation (EU) 2019/2152 of the European Parliament and of the Council on European business statistics repealing 10 legal acts in the field of business statistics. The Regulation sets the framework for the collection of R&D statistics and specifies the main variables of interest and their breakdowns at predefined level of detail. Commission Implementing Regulation (EU) No 2012/995 concerning the production and development of Community statistics on science and technology was in force until the end of 2020. 

Nature of the “obligations” of responsible national organisations to produce statistics and report to international organisations  

EKT is an agency and a national authority of the Hellenic

Statistical System (ELSS), responsible for the production of

national statistics for Research, Development and Innovation

(RDI) (see relevant decisions

https://www.statistics.gr/en/agencies ).

EKT is engaged in the production of RDI statistics since 2012

, taking the responsibility from the General Secretariat for

Research and Technology (relevant decision

Government Gazette (FEK) 1359/Β/25.04.2012).

From the beginning of its mandate and onwards, EKT is in

close collaboration with EL.STAT. More specifically, EKT

and ELSTAT are cooperating in the field of science and

technology statistics in accordance with the relevant mutually

signed MoUs (dated 28.01.2014, 04.06.2015, 15.12.2020 and 26.05.22).

EKT has recently participated as an ONA in the third round of ESS peer reviews and its compliance with the principles of the European Statistics Code of Practice was positively assessed.
6.1.2. National legislation
Existence of R&D specific statistical legislation  

The production of national R&D statistics is governed by

general national statistical legislation.
Legal acts  
  •  Greek Statistical Law No. 3832/2010, as in force.
  • Regulation on the Operation and Administration of the ELSTAT, 2012 (FEK 2390 B, 28.08.2012)
  • Regulation on the Statistical Obligations of the agencies of the ELSS (FEK 4083 Β, 20.12.2016)
All available in Greek only: http://metrics.ekt.gr/about ).
Obligation of responsible organisations to produce statistics (as derived from the legal acts)  

The production of national R&D statistics is governed by

general national statistical legislation.
Right of responsible organisations to collect data – obligation of (natural / legal) persons to provide raw and administrative data (as derived from the legal acts)  Both are covered by Greek Law 3832/2010.
Obligation of responsible organisations to protect confidential information from disclosure  (as derived from the legal acts)  

In line with the principles of the European Statistics Code of Practice in the frame of the Hellenic Statistical System.

Rights of access of third organisations / persons to data and statistics (as derived from the legal acts)  

Third parties (or persons), other than EL.STAT., are given access to aggregated data only.

Planned changes of legislation  No changes to our knowledge.
6.1.3. Standards and manuals

- Frascati Manual 2015, Guidelines for Collecting and Reporting Data on Research and Experimental Development

- European Business Statistics Methodological Manual on R&D Statistics

6.2. Institutional Mandate - data sharing

Not requested.


7. Confidentiality Top
7.1. Confidentiality - policy

Confidentiality, being one of the process quality components, concerns the privacy of data providers (households, enterprises, administrations and other respondents), the confidentiality of the information they provide and the extent of its use for statistical purposes.

A property of data indicating the extent to which their unauthorised disclosure could be prejudicial or harmful to the interest of the source or other relevant parties.

 

a)       Confidentiality protection required by law:

 

Confidentiality issues are clearly defined in the provisions on statistical confidentiality of the Greek statistical law (No. 3832/2010, as amended and in force), and are further specified in the Regulation on the Statistical Obligations of the Agencies of the ELSS.

As a National Authority Agency of the ELSS, EKT fully implements the above law and regulation as well as the European Statistics Code of Practice (principle 5 and relevant indicators). To this end, EKT has developed and published its Statistical Confidentiality Policy ( https://metrics.ekt.gr/sites/metrics-ekt/files/pages-pdf/EKT_Policy_Statistical_Confidentiality_1.1_en.pdf ).

 

b)       Confidentiality commitments of survey staff:

The internal personnel employed in the RDI statistics unit at EKT, the external statistical correspondents used for the collection and checking of primary data of its statistical surveys, as well as the external experts providing EKT with technical support or being assigned to carry out statistical works on account of EKT, commit themselves to the observance of statistical confidentiality of the data to which they have access or which they handle and sign a statistical confidentiality declaration.

7.2. Confidentiality - data treatment

Concerning the Procedures to identify confidential cells in data delivered to Eurostat, no confidential suppression/protection was applied on GOV data.


8. Release policy Top
8.1. Release calendar

Every year, during the first week of December EKT publishes a calendar of R&D Statistics press releases for the following year.

8.2. Release calendar access

 

The calendar is accessible by all users at the following link: Release Calendar | Metrics (ekt.gr)

8.3. Release policy - user access

EKT provides equal and simultaneous access to its statistical products to all users, as mentioned in the Dissemination Policy it applies

(EKT_Policy_Dissemination_1.1_en.pdf ).

EKT is fully complying with the relevant principles and regulations of the Statistical Confidentiality Policy. The main source of information for all R&D statistics derived by EKT, accessible to all users, is the following page: https://metrics.ekt.gr


9. Frequency of dissemination Top

Annually


10. Accessibility and clarity Top
10.1. Dissemination format - News release

See below.

10.1.1. Availability of the releases
  Availability (Y/N)1 Content, format, links, ...
Regular releases  Y  

Each R&D data publication is accompanied with press releases, which are sent to all media in Greece as well as social media (Twitter, Facebook and LinkedIn) and are published in EKT’ s website. For example: https://www.ekt.gr/el/news/28201  (provisional R&D results for 2021)

https://www.ekt.gr/el/news/29327 (final R&D results for 2021)
Ad-hoc releases    

1) Y - Yes, N – No

10.2. Dissemination format - Publications

See below.

10.2.1. Availability of means of dissemination
Means of dissemination Availability (Y/N)1 Content, format, links, ...
General publication/article

(paper, online)

 Y  

Dedicated webpage for R&D and Innovation statistics

http://metrics.ekt.gr/en has been created by EKT.

R&D data dissemination is made through different formats:

1) Publications (https://metrics.ekt.gr/research-development/publications): Publications that present the main findings of the R&D survey and in addition publications which analyze specific issues more in depth. Most of the publications are also available in English

https://metrics.ekt.gr/en/research-development/publications

2) Data brief reports: These are short publications that briefly analyze the results of specific topics: https://metrics.ekt.gr/en/research-development/articles

3) Articles in the magazine “Innovation, Research & Technology” published by EKT and sent by post to more than 5k recipients.

4) Articles in the e-newsletter “Research and Innovation” circulated by EKT to more than 50k recipients and promoted in social media (twitter, Facebook and LinkedIn). 
Specific paper publication (e.g. sectoral provided to enterprises)

(paper, online)

   

Final R&D results for 2021:

https://metrics.ekt.gr/en/node/666

1) Y – Yes, N - No 

10.3. Dissemination format - online database

Data tables (https://metrics.ekt.gr/research-development/datatables (available only in Greek)

10.3.1. Data tables - consultations

Not requested.

10.4. Dissemination format - microdata access

See below.

10.4.1. Provisions affecting the access
Access rights to the information  Microdata access is not provided to users outside EKT. Upon user requests, we produce more detailed analysis compared to the analysis of data requested and transmitted to Eurostat. This analysis is still in aggregated form. Data are protected for confidentiality and aggregations are produced in such way as not to reveal the identity of the enterprises/institution. 
Access cost policy  No charges.
Micro-data anonymisation rules  Microdata access is not provided to users outside EKT. Upon user requests, we produce more detailed analysis compared to the analysis of data requested and transmitted to Eurostat. This analysis is still in aggregated form. Data are protected for confidentiality and aggregations are produced in such way as not to reveal the identity of the enterprises/institution.
10.5. Dissemination format - other

See below.

10.5.1. Metadata - consultations

Not requested.

10.5.2. Availability of other dissemination means
Dissemination means Availability (Y/N)1  Micro-data / Aggregate figures Comments
Internet: main results available on the national statistical authority’s website  Y  

Aggregate/

figures
 https://metrics.ekt.gr/research-development/datatables
Data prepared for individual ad hoc requests  Y  

Aggregate/

figures
 Upon request from NSRF and RIS monitoring authorities, policy makers, expert group meetings, etc.
Other  

Aggregate/

figures
 

Data presented in the form of short articles: https://metrics.ekt.gr/research-development/articles

Data presented in conferences organised by EKT and targeting various audiences: Businesses, Researchers, academia, public e.tc. Conferences presentations can be accessed here: http://www.ekt.gr/el/events

1) Y – Yes, N - No 

10.6. Documentation on methodology

The production of R&D statistics follows the FM 2015 concepts, definitions and methodology as well as Eurostat "FM2015 Implementation, Harmonization EU Guidelines" as updated. A detailed handbook on R&D collection processes has been developed (internal) for GOV sector and is continuously enriched and improved.

National metadata (in Greek) are made available to all users in the dedicated EKT website:

https://metrics.ekt.gr/sites/metrics-ekt/files/pages-pdf/EKT_SIMS_RDstatistics_el.pdf

10.6.1. Metadata completeness - rate

Not requested.

10.7. Quality management - documentation

See below.

10.7.1. Information and clarity
Type(s) of data accompanying information available (metadata, graphs, quality reports, etc.)   

A glossary is available online to explain all the R&D related concepts to users of these statistics: https://metrics.ekt.gr/lexicon

 Reference material is also available online in the main page of the R&D statistics portal (English version: https://metrics.ekt.gr/en/research-development). This page contains information about the aim of the survey, links to reference documents  (Commission Regulation, FM, etc.), all related publications and datatables and the link for the online R&D questionnaire. Access to the online questionnaire requires username and password, which details are sent to respondents along with the (email) survey invitation.

Online helpdesk is also available for authorized users (i.e. respondents). However, the FAQ page is open to the public (http://helpdesk.metrics.ekt.gr).

National metadata are made available to Eurostat and are also published in the dedicated EKT website: https://metrics.ekt.gr/sites/metrics-ekt/files/pages-pdf/EKT_SIMS_RDstatistics_el.pdf

Publication with graphs, tables, maps, etc. are also made available, containing also methodological notes about the survey.
Request on further clarification, most problematic issues  No further requests for clarifications have been received.
Measure to increase clarity  Detailed publication with further analysis of results, events organization with stakeholders and key users, telephone and email support for clarifications, enhancing the FAQ page, etc.
Impression of users on the clarity of the accompanying information to the data   

Users of R&D statistics that participated to the user satisfaction survey, assessed the dedicated EKT website, which is used as the major dissemination stream for R&D statistics, to be of good or very good quality.
In 2021 the number of visits to the website reached almost 43k views.


11. Quality management Top
11.1. Quality assurance

EKT is an Agency of the Hellenic Statistical System (ELSS) and a National Authority, and as such it fully complies to the European and international standards concerning statistical methodologies, organizational procedures and IT infrastructure. EKT also complies strictly with the national and European legislative framework about statistics. EKT's quality policy is publicly available https://metrics.ekt.gr/en/policies .

EKT follows the Generic Statistical Business Process Model (GSBPM) for the production of RDI statistics. Accordingly, the workflow of a typical GOV R&D collection follows all level 1 phases of the GSBPM model and level 2 sub processes, modified to meet the specific sector and data collection requirements. A detailed handbook on the production of GOV R&D statistics, coupled with methodological annexes, has been developed and is continuously enriched and improved.

The quality of the data that EKT collects is controlled through a carefully implemented procedure that guarantees the production of meaningful statistics. In particular, the following practices are in place to enhance data quality:

Designing of the statistical process: Before the collection begins, a thorough investigation of the actions needed to ensure the quality of the data is conducted. This includes a) registry updates, b) questionnaire updates, c) preparation of the relevant infrastructure, d) preparation of a calendar program, and e) the employment of external statistical correspondents to assist the collection.

Data collection – start of the collection period: At the beginning of the collection period, a request to complete the questionnaire is forwarded electronically to all respondents, through an online questionnaire completion tool (LimeSurvey). The request is accompanied by an official letter by EKT’s Director, detailed instructions on how to complete the questionnaire, as well as instructions on how to request guidance regarding the completion process. For this purpose, EKT operates an electronic Help Desk which provides definitions, glossaries, and completion instructions with representative examples for each questionnaire. In addition, respondents can electronically submit questions and comments in the system which are, in turn, monitored by EKT members who are responsible for providing the relevant feedback. It is important to note that LimeSurvey provides statistics that assist the monitoring of the collection process. For administrative data, separate requests are sent via emails to the corresponding government entities.

Data Processing: After the end of the collection period, the micro-data are passed through several, and more sophisticated, validation layers. For the analysis process for R&D statistics, a Data Management System (DMS) is in place, along with peripheral analytics tools such as Python and R libraries. The validation process includes tests with respect to: a) logical rules not provided in the online questionnaire, b) the time-series component, c) ratios (e.g., expenditure over the number of FTEs, etc.), d) cross-testing with data reported from other countries cross-testing with administrative data from external (to EKT) sources, and e) statistical tests (e.g., identification of outliers).

The indicators production is automatically implemented via a combination of the DMS and R/Python libraries. Indicators are monitored for their validity through a second layer of tests based on the aggregated data. The validation process includes basic logical tests, time series tests as well as distribution tests (e.g., R&D and Innovation activities by region). Further, depending on their economic content, statistical outputs are additionally evaluated by EKT members with expertise on the field from other departments. The final SDMX file is tested and automatically corrected for rounding errors, through specific Python libraries.

11.2. Quality management - assessment

The overall quality of the GOV R&D statistical outputs is very good. The methodology has been designed in line with the FM recommendations, the relevant Commission Regulation and Eurostat guidelines. The continuous improvement is a key goal set by EKT and is implemented alongside the phases of the GSBPM model. Firstly, requirements of national users were met (such as the Hellenic Statistical Authority, the central /regional Monitoring Committees of the national development projects (NSRF projects) etc.).

The overall quality of the GOV R&D statistical outputs is very good. In 2015, EKT realized detailed case studies in 23 GOV institutions coming from all GOV R&D registry categories: Research Centers that are supervised by the General Secretariat for Research and Technology (GSRT), Other Public Research Institutions supervised by different Ministries, Entities supervised by the Ministry of Culture (Ephorates of Antiquities, museums, etc.), Public hospitals, and Special Accounts of Research Funds operating at regional health directorates. The case studies were performed through site-visits and interviews with respondents, the following topics were investigated:

  • the data collection methodology and the systems used by the institutions,
  • the problems faced as well as probable measurement errors,
  • best practices in data collection, and
  • motives for respondents to participate in the survey.

Moreover, any comments made by the respondents in the relevant section of the R&D questionnaire, regarding the questionnaire’s structure or the clarity of the guidelines provided, were considered. Overall, the respondents in GOV declared a satisfaction rate above 93%. Based on the results of the case studies as well as the respondents’ input, the structure of the online questionnaires was improved and the guidelines available to respondents through the RDI e-helpdesk operating at EKT, were enriched. For the RD survey round with reference year 2017, it is to be noted that the new methodological guidelines regarding the implementation of the FM 2015 were fully incorporated.

EKT’s R&D Information System is based on relevant international standards, such as CERIF and SDMX, robust technologies and best practices. The R&D Statistics Information System serves the objectives of: a) R&D micro-data collection, b) Workflow-based statistical analysis, c) Validation of data, d) R&D indicators production, e) Benchmarking analysis with third party datasets, f) Dissemination of R&D statistics.

The desired functionality is achieved by four subsystems, namely the Organisation Registry (OR), the Online Data Collection System (ODCS), the DMS and the SDMX Reference Implementation.

The Organisation Registry stores information about the institutions and organisations which are surveyed for collecting the R&D micro-data. In addition, it is used for managing access control and permissions for all services, including access to the online R&D surveys.

The Online Data Collection System (LimeSurvey) is where the R&D questionnaires reside and where the organisations are invited to login in order to participate in the survey. As a modern online survey tool, it covers a multitude of important requirements for conducting a survey, such as real time data validation, respondents’ management and the management of participants’ responses.

 The DMS is the single management point for all datasets involved (micro-data, paradata, organisation data, and indicators data), thus it is used to gather, store, interconnect and manage all collected R&D micro-data, the profiles of the organisations and the produced R&D indicators. Furthermore, it serves the following needs: data preservation and archiving (time-series), implementation of data validation & estimation workflows, real time automated generation of R&D indicators, data exporting (CSV, Excel, JSON etc.) and statistical reporting.


12. Relevance Top
12.1. Relevance - User Needs

See below.

12.1.1. Needs at national level
Users’ class1 Description of users Users’ needs
 1  Eurostat  

Production of European statistics

Data dissemination in Eurobase

https://ec.europa.eu/eurostat/web/science-technology-innovation/database and in various publications (http://ec.europa.eu/eurostat/web/science-technology-innovation/publications
 1  European Commission, other European agencies  


Policy making for R&D and
Innovation. Especially the R&D
intensity indicator is one of the
longstanding indicators for R&D in
Europe as well as one of the auxiliary indicators of the MIP Scoreboard.
Moreover, SDG 9 goal includes RD
intensity and RD personnel among its target indicators.
EKT provides the national indicators on gender equality and was statistical
correspondent for EC publication. See figures 2021.

 1  OECD  Policy making for R&D and Innovation Publications and studies (Science, Technology and Industry Scoreboard, STI Outlook, etc.) with country comparisons and presentation of country profiles.
 1  Hellenic Statistical Authority  Compilation of National Accounts in line with the revised European System of National and Regional Accounts (ESA 2010) (Commission Reg. 549/2013)
1 Greek Government R&D Intensity is one of the indicators of the National Reform Program. 
1 Ministry of Education, Religious Affairs and athletics, General Secretariat for Research and Innovation, other Ministries Ministry of Development, Regional Authorities, Central /regional Monitoring Committees of the NSRF national development projects etc. Policy making and national strategic planning for R&D and Innovation, Monitoring of EU strategic targets (e.g. EU2020), Monitoring and evaluation of NSRF National Development Frameworks (the current Partnership Agreement for the Development Framework 2014-2020 and of the Regional smart specialization policies (RIS) at national, regional and sectoral level.
1 National Council for Research, Technology and Innovation (NCRTI) Benchmarking, monitoring country's performance in Science and Technology and Innovation, monitoring GOV performance in Research, Technology and Innovation, evaluation and assessment of research outputs.
3 Media Country performance in relation to other European countries, publication of main policy R&D indicators.
4 Researchers, students  Analysis, subject-specific studies and/or regional studies, etc.

1)       Users' class codification

1- Institutions:
• European level: Commission (DGs, Secretariat General), Council, European Parliament, ECB, other European agencies etc.
• in Member States, at the national or regional level: Ministries of Economy or Finance, other ministries (for sectoral comparisons), National Statistical Institutes and other statistical agencies (norms, training, etc.), and
• International organisations: OECD, UN, IMF, ILO, etc.

2- Social actors: Employers’ associations, trade unions, lobbies, among others, at the European, national or regional level.

3- Media: International or regional media – specialized or for the general public – interested both in figures and analyses or comments. The media are the main channels of statistics to the general public.

4- Researchers and students (Researchers and students need statistics, analyses, ad hoc services, access to specific data.)

5- Enterprises or businesses (Either for their own market analysis, their marketing strategy (large enterprises) or because they offer consultancy services)

6- Other (User class defined for national purposes, different from the previous classes.)

12.2. Relevance - User Satisfaction

To evaluate if users' needs have been satisfied, the best way is to use user satisfaction surveys.

12.2.1. National Surveys and feedback
Conduction of a user satisfaction survey or any other type of monitoring user satisfaction  

Users’ workshops/meetings are organized with key stakeholders and policy makers (General Secretariat for Research and Innovation, Ministry of Education,  Religious Affairs and athletics, Ministry of Development and Investment, central /regional Monitoring Committees of the NSRF national development projects e.tc.) on a systematic base, at least twice a year.

Feedback is taken into consideration in the R&D survey design. Furthermore, in the dedicated website of metrics.ekt.gr, users can also add their feedback and ask questions (https://metrics.ekt.gr/feedback ). Also, this feedback is considered for future improvements and proposal of additional indicators.

A wide user satisfaction survey was conducted in 2015. The survey consisted of two parts: a) electronic questionnaire, and b) interviews with most important/ key users (e.g. officials from the General Secretariat for Research and Technology, key researchers).
User satisfaction survey specific for R&D statistics  

The users’ workshops are focused on R&D statistics: the main results are presented and explained with additional information and breakdowns relevant to the national environment.

 As regards the user survey, the questionnaire provided separate questions for each set of RDI statistics: R&D, GBARD and Innovation statistics.
Short description of the feedback received  Users are overall very satisfied with the quality of the statistics produced. As a result of the workshops, some additional breakdowns have been added to the R&D questionnaires (for example data collection on HCs / FTEs of R&D personnel working in projects financed by the National Development Framework). At a wider audience, user survey results showed that R&D indicators are used at least once every three months and they are considered very important.
12.3. Completeness

See below.

12.3.1. Data completeness - rate

Mandatory variables: 100%

12.3.2. Completeness - overview

Completeness is assessed via comparison of the data delivered against the requirements of Commission Implementing Regulation (EU) No 2020/1197 of 30 July 2020. The Regulation (EU) stipulates periodicity of variables that should be provided, breakdowns and if they should be provided mandatory or on voluntary basis.

 

  5

(Very Good)

4

(Good)

3

(Satisfactory)

2

 (Poor)

1

(Very poor)

Reasons for missing cells

Preliminary variables  x          
Obligatory data on R&D expenditure  x          
Optional data on R&D expenditure  x          
Obligatory data on R&D personnel  x          
Optional data on R&D personnel  x          
Regional data on R&D expenditure and R&D personnel  x          

Criteria:

A) Obligatory data. Only 'Very Good' = 100%, Poor' >95%; 'Very Poor' <100% apply.

B) Optional data. 'Very Good' = 100%; 'Good' = >75%; 'Satisfactory' 50 to 75%%; 'Poor' 25 to 50%; 'Very Poor' 0 to 25%.

12.3.3. Data availability

See below.

12.3.3.1. Data availability - R&D Expenditure
  Availability1 Frequency of data collection Gap years – years with missing data Modifications - Description Modifications - Year of introduction Modifications - Reasons
Source of funds  Y  Annual  2007  Additional break downs (internal / external, transfer exchange)   2020  2020_EL_RDI project Module 2.2 
Type of R&D  Y-2011  Biennial    Annual frequency  2020  
Type of costs  Y  Biennial  

2007,

2009 
 

-Annual frequency
-Additional break downs (Other current costs / External R&D personnel, Other current costs / Other costs, Capitalized computer software, Other intellectual property products)

 2020  2020_EL_RDI project Module 2.2
Socioeconomic objective  Y-2011  Biennial    Annual frequency  2020  
Region  Y-2011   Biennial    Annual frequency  2020  
FORD  Y-2011   Biennial    Annual frequency  2020  
Type of institution  Y-2020  Annual    Additional break downs (Central government, Regional (or state) government, Local (or municipal government, Non-profit institutions controlled by government)    2020_EL_RDI project Module 2.2
Economic activity Y-2020 Annual       2020_EL_RDI project Module 2.2

 

 

 

1) Y-start year, N – data not available

12.3.3.2. Data availability - R&D Personnel (HC)
  Availability1 Frequency of data collection Gap years – years with missing data Modifications - Description Modifications - Year of introduction Modifications - Reasons
Sex  Y-1999  Biennial  

2007,
2009

 Annual frequency  2011  
Function  Y-1993  Biennial  

2007,
2009

  Annual frequency  2011  
Qualification  Y-1993  Biennial  

2007,
2009

 Annual frequency  2011  
Age  Y-2011  Biennial    

Available for researchers only.
Annual frequency after 2020

 2011,2020  
Citizenship  Y-2011  Biennial    

Available for researchers only.
Annual frequency after 2020

  2011,2020  
Region  Y-2011  Biennial    Annual frequency after 2020  2020  
FORD  Y-2011   Biennial    Annual frequency after 2020  2020  
Type of institution  Y-2011  Annual        
Internal/External Y-2020 Annual       2020_EL_RDI project Module 2.1

1) Y-start year, N – data not available

12.3.3.3. Data availability - R&D Personnel (FTE)
  Availability1 Frequency of data collection Gap years – years with missing data Modifications - Description Modifications - Year of introduction Modifications - Reasons
Sex  Y-1999  Biennial  2009  Annual frequency  2011
 
Function  Y-1993  Biennial  2009   Annual frequency   2011  
Qualification   Y-1993  Biennial  2009    Annual frequency    2011  
Age  N          
Citizenship          
Region  Y-2011  Biennial    Annual frequency  2020  
FORD  Y-2011  Biennial    Annual frequency  2020  
Type of institution  Y-2011  Annual    Annual frequency  2020  
Internal/External Y-2020 Annual       2020_EL_RDI project Module 2.1

1) Y-start year, N – data not available

12.3.3.4. Data availability - other
Additional dimension/variable available at national level1) Availability2  Frequency of data collection Breakdown variables Combinations of breakdown variables Level of detail
 R&D Expenditure funded by GOV  Y-2011  Annual  a) Funds from the Ordinary Government Budget, b) Funds from the NSRF National Development Framework, c) Public Investment Program (PIP) other than ESPA, d) Other (Regional Authorities, Municipalities, Special accounts, etc.)    
 Intramural R&D expenditure in specific fields of interest (and percentage financed by the government)  Y-2017  Annual  Fields of Research and Innovation Strategy for Smart Specialisation (RIS 3) in the following eight fields: Agri-food, Health – medicines, ICT, Energy, Environment and sustainable development, Transport, Materials – construction, Tourism – Culture – Creative industries.    
 Extramural R&D expenditure  Y-2013  Annual  Type of organization undertaken the R&D activity (private /public, national / abroad): a) Domestic enterprises (excl. banks and state-owned enterprises), b) Domestic banks, c) Domestic enterprises of the public sector, d) Domestic non-profit Institutions, e) Domestic other institutions of the public sector, f) Foreign enterprises, g) Other foreign institutions    

1) This question is optional. It refers to variables and breakdowns NOT asked by the Commission Implementing Regulation (EU) No 2020/1197 (neither as 'optional').

2) Y-start year


13. Accuracy Top
13.1. Accuracy - overall

Accuracy in the statistical sense denotes the closeness of computations or estimates to the exact or true values. Statistics are not equal with the true values because of variability (the statistics change from implementation to implementation of the survey due to random effects) and bias (the average of the possible values of the statistics from implementation to implementation is not equal to the true value due to systematic effects).

 

Several types of statistical errors occur during the survey process. The following typology of errors has been adopted:

1. Sampling errors. These only affect sample surveys. They are due to the fact that only a subset of the population, usually randomly selected, is enumerated.

2. Non-sampling errors. Non-sampling errors affect sample surveys and complete enumerations alike and comprise:

a) Coverage errors,

b) Measurement errors,

c) Non response errors and

d) Processing errors.

 

Model assumption errors should be treated under the heading of the respective error they are trying to reduce.

13.1.1. Accuracy - Overall by 'Types of Error'
  Sampling errors Non-sampling errors1) Model-assumption Errors1) Perceived direction of the error2)
Coverage errors Measurement errors Processing errors Non response errors
Total intramural R&D expenditure -  1  4  3    +/-
Total R&D personnel in FTE    +/-
Researchers in FTE    +/-

1)  Ranking of the type(s) of errors that result in over/under-estimation, from the most important source of error (1) to the least important source of error (5). In the event that errors of a particular type do not exist, is used the sign ‘-‘.

2)  The perceived direction of the ‘overall’ error using the signs “+” for over estimation, “-” for under estimation and “+/-” when assumption of the direction of the error cannot be made for R&D.

13.1.2. Assessment of the accuracy with regard to the main indicators
Indicators 5

(Very Good)1

4

(Good)2

3

(Satisfactory)3

2

(Poor)4

1

(Very poor)5

Total intramural R&D expenditure  x        
Total R&D personnel in FTE  x        
Researchers in FTE  x        

1) 'Very Good' = High level of coverage (annual rate of substitution in the target population lower than 5%). High average rates of response (>80%) in census and sample surveys. Full data consistency with reference to totals and relationships between variables in the dataset sent to Eurostat.  

2) 'Good' = In the event that at least one out of the three criteria described above would not be fully met.

3) 'Satisfactory' = In the event that the average rate of response would be lower than 60% even by meeting the two remaining criteria.

4) 'Poor' = In the event that the average rate of response would be lower than 60% and at least one of the two remaining criteria would not be met.

5) 'Very Poor' = If all the three criteria are not met.

13.2. Sampling error

That part of the difference between a population value and an estimate thereof, derived from a random sample, which is due to the fact that only a subset of the population is enumerated.

13.2.1. Sampling error - indicators

The main indicator used to measure sampling errors is the coefficient of variation (CV). 
Definition of coefficient of variation: 
CV= (Square root of the estimate of the sampling variance) / (Estimated value)

13.2.1.1. Variance Estimation Method

-

13.2.1.2. Coefficient of variation for R&D expenditure by source of funds
Source of funds R&D expenditure
Business enterprise  Not applicable.
Government  Not applicable.
Higher education  Not applicable.
Private non-profit  Not applicable.
Rest of the world  Not applicable.
Total  Not applicable.
13.2.1.3. Coefficient of variation for R&D expenditure by function and qualification
    R&D personnel (FTE)
Function Researchers  Not applicable.
Technicians  Not applicable.
other support staff  Not applicable.
Qualification ISCED 8  Not applicable.
ISCED 5-7  Not applicable.
ISCED 4 and below  Not applicable.
13.3. Non-sampling error

Non-sampling errors occur in all phases of a survey. They add to the sampling errors (if present) and contribute to decreasing overall accuracy. It is important to assess their relative weight in the total error and devote appropriate resources for their control and assessment.

13.3.1. Coverage error

Coverage errors are due to divergences between the target population and the frame population. The frame population is the set of target population members that has a chance to be selected into the survey sample. It is a listing of all items in the population from which the sample is drawn that contains contact details as well as sufficient information to perform stratification and sampling.

 

a)       Description/assessment of coverage errors :

 There are minor divergences between target and frame population.

 

b)      Measures taken to reduce their effect:

 

Not applicable.

 

c)       Share of PNP (if PNP is included in GOV):

 

PNP is not included in GOV.

13.3.1.1. Over-coverage - rate

Not requested.

13.3.1.2. Common units - proportion

Not requested.

13.3.2. Measurement error

Measurement errors occur during data collection and generate bias by recording values different than the true ones (e.g. difficulty to distinguish intramural from extramural R&D Expenditure). The survey questionnaire used for data collection may have led to the recording of wrong values, or there may be respondent or interviewer bias.

 

a)       Description/assessment of measurement errors:

 

The main difficulties that have been reported by respondents concerned a) the separation of R&D from other activities, b) the separation of in-house R&D performance from outsourcing activities, c) the breakdown of labor cost in all kinds of personnel.

 

b)      Measures taken to reduce their effect:

 

  • Guidelines have been included in the questionnaire and both electronic and telephone helpdesk

was operating throughout the collection period to respond to enquiries by the respondents.

  • Moreover, in cases where measurement errors have been detected during the validation phase (e.g., inconsistencies between the personnel and expenditure data), institutions have been contacted by experienced staff to clarify misunderstandings, etc.
  • In the electronic questioner a set of validation rules is added in order to help the respondents to complete the questioner. Also, a set of further information, guidelines and examples added in the electronic questioner in order to explain more the questions.
  • A detailed question with the breakdown of the personnel is added in order to allow respondents to clearly report all the categories of personnel (external, internal, full-time, part-time, etc.) and its breakdowns HCs, FTEs, Education level, Age group, Nationality.
  • A detailed question about the labor costs of all personnel categories as explained above, which are then compared to the respective FTEs to assure the consistency of the reported figures.
  • A mandatory question relevant to R&D activities that can be answered as 'Yes/No' in order to proceed to the next section. The answers to these questions help respondents understand the concept of R&D and leads to the identification of R&D activities in their organization / enterprise.
  • Regional analysis is also separated by type of personnel (researchers and other staff) for HCs / FTEs as also R&D expenditures by region. Users have to report except the R&D expenditures per region also the HCs and FTEs separated by sex.
13.3.3. Non response error

Non-response occurs when a survey failed to collect data on all survey variables from all the population units designated for data collection in a sample or complete enumeration.

There are two elements of non-response:

-Unit non-response which occurs when no data (or so little as to be unusable) are collected on a designated population unit.

-Item non-response which occurs when data only on some, but not all survey variables are collected on a designated population unit.

The extent of response (and accordingly of non response) is also measured with response rates. 

13.3.3.1. Unit non-response - rate

The main interest is to judge if the response from the target population was satisfactory by computing the un-weighted response rate.

Definition: Eligible are the survey units which indeed belong to the target population. Frame imperfections always leave the possibility that some units may not belong to the target population. Moreover, when there is no contact with certain units and no other way to establish their eligibility they are characterised as ‘unknown eligibility units’.

Un-weighted Unit Non- Response Rate = 1 - (Number of units with a response) / (Total number of eligible and unknown eligibility units in the survey) 

13.3.3.1.1. Un-weighted unit non-response rate
Number of units with a response in the survey Total number of units in the survey Unit non-response rate (Un-weighted)
 321  336  

4.46%
(All of the non-responding units belong
to the perimeter of the GOV R&D
Directory. The core GOV
institutions with systematic R&D had a
98.5% response rate.)

13.3.3.2. Item non-response - rate

Definition:
Un-weighted Item Non-Response Rate (%) = 1-(Number of units with a response for the item) / (Total number of eligible , for the item, units in the sample) * 100

13.3.3.2.1. Un-weighted item non-response rate
R&D variable/breakdown Item non-response rate (un-weighted) (%) Comments
 -  -  NOT APPLICABLE
     
     
13.3.3.3. Measures to increase response rate

There was extensive communication with directors of the GOV institutions as well as official bodies (such as the Plenary Meetings of the Presidents of Research Centers). E-mail invitations, accompanied with an official letter, signed by the Director of EKT, were sent to launch the survey. The e-mail invitations explained the purpose and mandatory nature of the survey.

The appropriate links to the online publications of EKT were also provided in the invitations to help respondents to better understand the use of the data they provide. Hard copies of the publications were also sent via post.

Statistical units were urged to respond via email reminders as well as systematic follow-up by phone and personal emails.

13.3.4. Processing error

Between data collection and the beginning of statistical analysis, data must undergo a certain processing: coding, data entry, data editing, imputation, etc. Errors introduced at these stages are called processing errors. Data editing identifies inconsistencies or errors in the data.

13.3.4.1. Identification of the main processing errors
Data entry method applied  No data entry errors since the online questionnaire has been used by all respondents.
Estimates of data entry errors  Not applicable.
Variables for which coding was performed  Not applicable.
Estimates of coding errors  Not applicable.
Editing process and method  Validation checks were embedded into the online questionnaire to inform users, in real time, about the occurrence of errors (check totals, sub-totals, totals between questions, no of FTEs vs no of HCs). In addition, extra controls were applied on the submitted data, using statistical software to to cover other types of errors. Identified errors have been corrected as described below.
Procedure used to correct errors  In cases of 'important' errors respondents have been contacted by phone for correction/verification. In cases of less 'important' errors, values have been corrected with logical assumptions and imputation.
13.3.5. Model assumption error

Not requested.


14. Timeliness and punctuality Top
14.1. Timeliness

Timeliness and punctuality refer to time and dates, but in a different manner: the timeliness of statistics reflects the length of time between their availability and the event or phenomenon they describe. Punctuality refers to the time lag between the release date of the data and the target date on which they should have been delivered, with reference to dates announced in the official release calendar.

14.1.1. Time lag - first result

Time lag between the end of reference period and the release date of the results:
Indicator: (Release date of provisional/ first results) - (Date of reference for the data)

 

a) End of reference period: 

December 2021 (T)

b) Date of first release of national data: 

October 2022 (T+10)

c) Lag (days): 

300 (10 months)

14.1.2. Time lag - final result

a) End of reference period: 

December 2021 (T)

b) Date of first release of national data: 

June 2023 (T+18)

c) Lag (days): 

540 (18 months)

14.2. Punctuality

Punctuality refers to the time lag between the release date of data and the target date on which they were scheduled for release as announced officially.

14.2.1. Punctuality - delivery and publication

Punctuality of time schedule of data release = (Actual date of the data release) - (Scheduled date of the data release)

14.2.1.1. Deadline and date of data transmission
  Transmission of provisional data Transmission of final data
Legally defined deadline of data transmission (T+_ months) 10 18
Actual date of transmission of the data (T+x months)  10  18
Delay (days) 
Reasoning for delay


15. Coherence and comparability Top
15.1. Comparability - geographical

See below.

15.1.1. Asymmetry for mirror flow statistics - coefficient

Not requested.

15.1.2. General issues of comparability

No deviations from FM recommendations and classifications. Therefore, R&D data for Greece are considered to be comparable with international R&D data.

15.1.3. Survey Concepts Issues

The following table lists a number of key survey concepts and conceptual issues; it gives reference to the Commission Implementing Regulation (EU) No 2020/1197, Frascati manual and EBS Methodological Manual on R&D Statistics paragraphs with recommendations about these concepts / issues.

 

Concept / Issues Reference to recommendations Deviation from recommendations Comments on national definition / Treatment – deviations from recommendations
R&D personnel FM2015 Chapter 5 (mainly paragraph 5.2).  No deviation.
 
Researcher FM2015, § 5.35-5.39.  No deviation.  
Approach to obtaining Headcount (HC) data FM2015, § 5.58-5.61 (in combination with  Eurostat's EBS Methodological Manual on R&D Statistics).  No deviation.  
Approach to obtaining FTE data FM2015, § 5.49-5.57 (in combination with Eurostat's EBS Methodological Manual on R&D Statistics).  No deviation.  

Starting from the 2017 survey, we provide detailed guidelines to R&D performing units to exclude personnel with less than 0.1 FTE of R&D activity. We also implement additional validation rules in the online questionnaire to ensure that these persons are not recorded.
Total R&D personnel is divided into internal and external personnel. In the questionnaire we implement a more detailed division of R&D
personnel into five groups, of which the first two groups (a and b) represent the internal personnel:
a) regular personnel,
b) temporary personnel,
c) external associates,
d) Greek University personnel and
e) external associates who are paid
by another organization/enterprise. This detailed breakdown also improves the understanding of respondents.

 

Reporting data according to formula: Total R&D personnel = Internal R&D personnel + External R&D personnel FM2015, §5.25  No deviation.  
Intramural R&D expenditure FM2015, Chapter 4 (mainly paragraph 4.2).  No deviation.  
Statistical unit FM2015, § 8.64-8.65 (in combination with Eurostat's EBS Methodological Manual on R&D Statistics).  No deviation.  
Target population FM2015, § 8.63 (in combination with Eurostat's EBS Methodological Manual on R&D Statistics).  No deviation.  
Sector coverage FM2015, § 8.2-8.13 (in combination with Eurostat's EBS Methodological Manual on R&D Statistics).  No deviation.  
Hospitals and clinics FM2015, § 8.22 and 8.34  No deviation.  
Borderline research institutions FM2015, § 8.14-8.23 (in combination with Eurostat's EBS Methodological Manual on R&D Statistics).  No deviation.  
Fields of research & development coverage and breakdown Reg. 2020/1197 : Annex 1, Table 18  No deviation.  
Socioeconomic objectives coverage and breakdown Reg. 2020/1197 : Annex 1, Table 18  No deviation.  
Reference period Reg. 2020/1197 : Annex 1, Table 18  No deviation.  
15.1.4. Deviations from recommendations

The following table lists a number of key methodological issues, which may affect the international comparability of national R&D statistics. The table gives the references in the Frascati manual, where related recommendations are made. Countries are asked to report on the existence of any deviations from existing recommendations and comment upon.

Methodological issues Deviation from recommendations Comments on national treatment / treatment deviations from recommendations
Data collection method  No deviation.  
Survey questionnaire / data collection form  No deviation.  
Cooperation with respondents  No deviation.  
Data processing methods  No deviation.  
Treatment of non-response  No deviation.  
Variance estimation    Not applicable.
Data compilation of final and preliminary data  No deviation.  
15.2. Comparability - over time

See below.

15.2.1. Length of comparable time series

See below.

15.2.2. Breaks in time series
  Length  of comparable time series  Break years1 Nature of the breaks
R&D personnel (HC)   9 consequent years, starting from 2011.  2011, 1983  2011: Coverage of GOV has been extended to also cover public hospitals as well as all institutions administered by the Ministry of Culture (mainly archaeological and cultural institutions). 1983: The methodology for identifying R&D personnel was brought into line with the recommendations of the FΜ, resulting in changes to personnel cost.
  Function      
  Qualification      
R&D personnel (FTE)  9 consequent years, starting from 2011.  2011, 1983  2011: Coverage of GOV has been extended to also cover public hospitals as well as all institutions administered by the Ministry of Culture (archaeological and cultural institutions).
 1983: The methodology for identifying R&D personnel was brought into line with the recommendations of the Frascati Manual, resulting in changes to personnel cost.
  Function      
  Qualification      
R&D expenditure  9 consequent years, starting from 2011.  2011  2011: Coverage of GOV has been extended to also cover public hospitals as well as all institutions administered by the Ministry of Culture (archaeological and cultural institutions).
Source of funds      
Type of costs      
Type of R&D      
Other      

1)       Breaks years are years for which data are not fully comparable to the previous period.

15.2.3. Collection of data in the even years

All data for R&D personnel (HC, FTE) and Expenditure are collected annually (odd and even years).

15.3. Coherence - cross domain

This part deals with any national coherence assessments which may have been undertaken. It reports results for variables which are the same or relevant to R&D statistics, from other national surveys and / or administrative sources and explains and comments on their degree of agreement with R&D statistics.

15.3.1. Coherence - sub annual and annual statistics

Not requested.

15.3.2. Coherence - National Accounts

Based on the MOU between ELSTAT and EKT, microdata are sent annually to ELSTAT for inclusion in the National Accounts.

15.3.3. National Coherence Assessments
Variable name R&D Statistics - Variable Value Other national statistics - Variable value Other national statistics - Source Difference in values (of R&D statistics) Explanation of / comments on difference
           
           
           
           
           
           
15.4. Coherence - internal

See below.

15.4.1. Comparison between preliminary and final data

This part compares key R&D variables as preliminary and final data.

 

  Total R&D expenditure – GOVERD (in 1000 of national currency) Total R&D personnel (in FTEs) Total number of researchers  (in FTEs)
Preliminary data (delivered at T+10)  579,004.35  14,813.90  8,537.05
Final data (delivered T+18)  590,319.07  15,254.22  8,823.80
Difference (of final data)  11,314.72  440.32  286.75
15.4.2. Consistency between R&D personnel and expenditure
  Average remuneration (cost¨in national currency)
Consistency between FTEs of internal R&D personnel and R&D labour costs (1)  25,582.26
Consistency between FTEs of external R&D personnel and other current costs for external R&D personnel (2)  20,121.58

(1)    Calculate the average remuneration (cost) of individuals belonging to the internal R&D personnel, excluding those who are only formally ‘employees’ (university students, grant holders, etc.).

(2)    Calculate the average remuneration (cost) of individuals belonging to the external R&D personnel (FTEs/other current R&D costs for external R&D personnel).


16. Cost and Burden Top

The assessment of costs associated with a statistical product is a rather complicated task since there must exist a mechanism for appointing portions of shared costs (for instance shared IT resources and dissemination channels) and overheads (office space, utility bills etc). The assessment must become detailed and clear enough so that international comparisons among agencies of different structures are feasible. 

16.1. Costs summary
  Costs for the statistical authority (in national currency) % sub-contracted1)
Staff costs Not available.  
Data collection costs Not available.  
Other costs Not available.  
Total costs Not available.  
Comments on costs
 

1)       The shares of the figures given in the first column that are accounted for by payments to private firms or other Government agencies.

16.2. Components of burden and description of how these estimates were reached
  Value Computation method
Number of Respondents (R)  2.3  

Information has been retrieved from a relevant question that is included in the questionnaire.
Number of Respondents is calculated as the average persons per unit.

Average Time required to complete the questionnaire in hours (T)1  10.9  

Information has been retrieved from a relevant question that is included in the questionnaire.

Average time is calculated as the average amount of hours needed to complete the survey, as reported by all responding units.

Average hourly cost (in national currency) of a respondent (C)  N/A  
Total cost  N/A  

1)        T = the time required to provide the information, including time spent assembling information prior to completing a form or taking part in interview and the time taken up by any subsequent contacts after receipt of the questionnaire (‘Re-contact time’)


17. Data revision Top
17.1. Data revision - policy

Not requested.

17.2. Data revision - practice

Not requested.

17.2.1. Data revision - average size

Not requested.


18. Statistical processing Top
18.1. Source data

Several separate activities are used for the collection of raw data or pre-compiled administrative data and statistics related to R&D. For simplicity, we call them surveys irrespective of whether they are sample surveys, censuses, collections of administrative data/pre-compiled statistics. This section presents the names of the surveys by sector of performance as well as methodological information for each survey. Depending on the type of survey and sector of performance, only the sections corresponding to that survey and sector are filled in.

18.1.1. Data source – general information
Survey name  GOV R&D survey Personnel engaged and Expenditure spent on R&D activities.
Type of survey   Census to all known or potential R&D performing units in the GOV.
Combination of sample survey and census data  
Combination of dedicated R&D and other survey(s)  

Direct census survey has been used for almost all GOV entities

Especially for the entities supervised by the Ministry of Culture as well as public hospitals, with continuous and systematic R&D activities, a combination of census survey and administrative data collection from the Ministry of Culture and the Ministry of Health was used. Administrative data refer to the institutional funding by the Ordinary Budget allocated to the above entities. R&D activities funded by all other sources of funds (i.e. R&D projects financed by NSRF national programs, by EU, by companies, etc.), were provided through the direct census survey by the entities supervised by the Ministry of Culture and by the Special Accounts of Research Funds operating at regional health directorates (public hospitals).

Additional administrative sources (MIS information, eCORDA,) as well GBARD data were used for validation /imputation purposes.
    Sub-population A (covered by sampling)  
    Sub-population B (covered by census)  

265 core institutions with systematic R&D activities.

•  71 institutions with potential or occasional R&D activities.
Variables the survey contributes to  All R&D variables requested by the Commission Regulation No 995/2012.
Survey timetable-most recent implementation  

All R&D variables requested by the Commission Regulation No 995/2012.
2011 survey: EKT took over the collection and production of R&D and GBAORD statistics in spring 2012. The GBAORD collection preceded the R&D survey. Within this limited time frame, the GOV survey started in the beginning of 2013 and ended in May 2013.
2013 survey: The GOV survey started in July 2014 and ended end of October 2014. During 2015, additional information has been collected, via personalised communication, to verify data for 2014 reference year, when needed.
2015 survey: The GOV survey started in March 2016 and ended in September 2016.
2017 survey: The GOV survey started end of May 2018 and finished in September 2018
2019 survey: The GOV survey started 1st of June 2020 and ended in September 2020.
2021 survey: The GOV survey started 1st of March 2022 and ended in September 2022.

18.1.2. Sample/census survey information
  Stage 1 Stage 2 Stage 3
Sampling unit  Government institutions (public research centers, archaeological and cultural institutions, public hospitals, agricultural institutions, etc.).    
Stratification variables (if any - for sample surveys only)  No stratification – census survey    
Stratification variable classes  Not applicable.    
Population size  

336 government institutions.

   
Planned sample size  Not applicable.    
Sample selection mechanism (for sample surveys only)  Not applicable.    
Survey frame  Survey frame comprises all GOV institutions that are included in the Directory of Greek R&D organizations developed by EKT. The directory comprises all gov institutions systematically performing R&D activities (e.g. Greek research centers) and also those that potentially or occasionally perform R&D. The Directory is regularly updated through the statistical Register of General Government Entities that is maintained by ELSTAT which is updated annually to reflect changes in the perimeter of General Government, such as the inclusion of new entities, which are classified into General Government according to the ESA95 criteria as well as with the deletion of entities that either are abolished or cease to fulfill the criteria for remaining in the General Government sector.    
Sample design Not applicable.    
Sample size Not applicable.    
Survey frame quality Overall assessment is very good.    
18.1.3. Information on collection of administrative data or of pre-compiled statistics
Source The following sources are used: a) Detailed administrative information at institution level for the institutions supervised by from the Ministry of Education, Religious Affairs and athletics the Ministry of Culture, and the Ministry of Health as well as GBAORD detailed data, b) Monitoring Information System (M.I.S.) including information about projects co-financed under the NSRF, c) eCORDA database with information about signed grants/beneficiaries with regards to EU research programs such as Horizon 2020,d) information provided by units that administer research projects (ELKEA) inside the six Health Region Administrations that operate in Greece for monitoring and managing private funding of clinical trials (primarily coming from pharmaceutical companies) conducted in public hospitals.
Description of collected data / statistics  R&D expenditure, where available. Total expenditures and total personnel where coefficients are applied to isolate the R&D component.
Reference period, in relation to the variables the survey contributes to  Administrative data is available for reference period 2021.
18.2. Frequency of data collection

See 12.3.3.

18.3. Data collection

See below.

18.3.1. Data collection overview
Information provider  

1. GOV R&D survey R&D Expenditure and Personnel 2021: Data collection at
micro level. Data were provided by GOV institutions which filled in the online
questionnaires.
2. Detailed administrative information by the Ministry of Culture: Data collection at micro level. Data was provided, at institution level.
3.Survey to Special Accounts of Research Funds operating at regional health
directorates: Data collection at micro level. Data was provided -via questionnaires by the Special Accounts of Research Funds operating at the seven regional health
directorates of the country.
Detailed administrative information:
4. Data collection at micro level. Data was provided, at institution level, by the
Ministry of Health.
5. MIS / NSRF: Data collection at micro level. Data was provided at project
/beneficiary level by the central Special Service for the
MIS and by the NSRF monitoring service operating at the General
Secretariat of Research and Technology.
6. eCORDA: Data collection at micro level. EKT has access to eCORDA databases for downloads of raw data at project / beneficiary level.

Description of collected information  

1. Information requested by the Regulation and additional/ more detailed breakdowns.

2. Detailed information, at institution level, for the personnel and expenditures

funded by the Ordinary Budget (institutional funding) for the entities supervised by the Ministry of Culture. This data supplement the R&D data collected from the survey for these particular entities.

3. Administrative data collection. Detailed information, at institution level, for the personnel and expenditures funded by the Ordinary Budget (institutional funding) for the public hospitals that have systematic R&D activities.

4. Information about R&D projects financed by national development frameworks (ESPA 2014-2020) at project /beneficiary level. The information has been used for validation / imputation purposes.

5. Information about R&D projects financed by EU research programs

(Horizon 2020, FP7) at project /beneficiary level. The information has been used for validation / imputation purposes.
Data collection method  

Data are collected with the use of online questionnaires filled out by the
institutions. EKT has developed a special platform (LimeSurvey) for the
implementation of the survey. Administrative data are collected for the
institutions supervised by the Ministry of Education, Religious Affairs and athletics, the Ministry of Culture, and the Ministry of Health as well as GBAORD detailed. Administrative data are checked by the EKT before their use in statistical indices production.

Time-use surveys for the calculation of R&D coefficients  The latest time-use survey for the calculation of R&D coefficients of public hospitals took place in 2015.
Realised sample size (per stratum)  
Mode of data collection (face-to-face interviews; telephone interviews; postal surveys, etc.)  Since 2012 (reference year 2011 onwards), this is an online survey. Every institution received, via e-mail, its ‘personal’ log-in details to access the online questionnaire. Institutions have the possibility to preview the questionnaire before completion. Frequently, EKT aids the institutions for the completion of the questionnaire via telephone interviews. Finally, several meetings have been realized with the directors of the institutions so as to further explain R&D concepts and variables in the questionnaire.
Incentives used for increasing response  Extensive communication with directors of institutions as well as official bodies (such as the Meetings of the Presidents of Research Centers). The survey is launched with an official letter, signed by the Director of EKT, that is attached to the e-mail invitations so as to explain the purpose and mandatory nature of the survey. The appropriate links to the online publications of EKT are also provided in the invitations to help respondents to better understand the use of the data they provide. Hard copies of the publications are also sent via post to the core group of systematic R&D performers. 
Follow-up of non-respondents  Several email reminders. Systematic follow-up by phone and personal emails.
Replacement of non-respondents (e.g. if proxy interviewing is employed)  No replacement.
Response rate (ratio of completed "interviews" over total number of eligible enterprises or enterprises of unknown eligibility)  

72% (2011) - Since this was the first time to run the survey after reference year 2005, our frame population covered exhaustively all government units that potentially have R&D activities. Most of the non-responding units (e.g. libraries) had no R&D activities and they are included in our GOV frame population as potential /occasional R&D performers.
98.3% for core institutions (2013)
100% for the core institutions (2015)
100% for the core institutions (2017)
100% for the core institutions (2019)
99.4% for the core institutions (2020)
98.5% for the core institutions (2021)

Non-response analysis (if applicable -- also see section 18.5.4 Data compilation - Weighting and Estimation methods)  Not applicable
18.3.2. Questionnaire and other documents
Annex Name of the file
R&D national questionnaire and explanatory notes in English:  
R&D national questionnaire and explanatory notes in the national language:  GOV_questionnaire_2021_EL.pdf
Other relevant documentation of national methodology in English:  
Other relevant documentation of national methodology in the national language:  
18.4. Data validation

Validation processes run throughout all phases of the R&D survey. All institutions belonging to the target population (GOV R&D Directory at EKT) are fully covered by a census survey. In order to eliminate any important missing information, an essential requirement of the R&D survey is to attain a 98.5% response rate by the core institutions of GOV R&D Directory, namely all GOV institutions which have a systematic R&D activity and contribute the major part of the final data (265 institutions). The administrative data that is used for the compilation of R&D data as well as for validation / imputation purposes come from official reliable sources at national (governmental / other authorities) or European level (eCORDA database). To ensure the validity of the data as completed by the respondents, validation checks are embedded into the online questionnaire to inform users, in real time, about the occurrence of errors (check totals, sub-totals, totals between questions, no of FTEs larger than no of HCs e.tc.). Any inconsistencies in the reported data are checked in each questionnaire received, namely at the institutional level, as for example FTEs of internal R&D personnel vs R&D labour costs considering the average remuneration cost for each sector, proportion of other current costs vs labour costs etc. Time series are also checked at the level of the each responding unit and item peaks are crosschecked both with respondents and administrative sources (for example a significant increase in EC funding for a reporting unit is checked with the eCORDA database and communicated and verified by the respondent). Thorough validation is carried out to check the coherence of the outputs produced. To this end, in addition to the extensive statistical checking, multiple official sources are used (as explained above, MIS data, eCORDA, etc.) to check the collected data against relevant data. The final outputs are interpreted using both tangible and tacit knowledge accumulated at EKT as well as sectoral studies produced by other national bodies.

18.5. Data compilation

See below.

18.5.1. Imputation - rate

No imputation was used for GOV 2021.

18.5.2. Data compilation methods
Data compilation method - Final data (between the survey years)   Not applicable – the R&D survey is conducted every year.
Data compilation method - Preliminary data   Not applicable – the R&D survey is conducted every year.
18.5.3. Measurement issues
Method of derivation of regional data  Questionnaires for all sectors included separate questions for the regional element of R&D Personnel and R&D expenditure. Respondents are asked to distribute total R&D personnel (headcounts and FTE by sex), Researchers (headcounts and FTE by sex) and total intramural expenditure into the 13 Greek regions (NUTS2).
Coefficients used for estimation of the R&D share of more general expenditure items  

In the GOV sector and based on the information available, coefficients have been applied in some cases where R&D is part of the institutions activities funded by the ordinary budget.
The purpose is to reduce response burden and at the same time increase data
reliability.
The data quality is very good. No R&D coefficients are applied for the
estimation of R&D activities funded by other sources than the Ordinary
Budget (such as EC, NSRF, etc.) for which data are collected directly by the responding GOV units.

Inclusion or exclusion of VAT and provisions for depreciation in the measurement of expenditures  In line with the FM, respondents were asked to exclude VAT and depreciation from R&D expenditures.
Differences between national and Frascati Manual classifications not mentioned above and impact on national statistics  No deviations from FM classifications.
18.5.4. Weighting and estimation methods
Description of weighting method   No weighting has been applied in GOV.
Description of the estimation method  Estimations for unit non-response were made using data from official (administrative) sources.  
18.6. Adjustment

Not requested.

18.6.1. Seasonal adjustment

Not requested.


19. Comment Top


Related metadata Top


Annexes Top