Expert Group on Indicators

Indicators for Researchers' Engagement with Open Science and its Impacts

How can the responsible engagement of the scientific communities with open knowledge practices be stimulated? In what way may current evaluation protocols hinder the development of open science and scholarship? Which new indicators can be developed to ensure that the potential of open research is realized? And how can they contribute to a more robust and high quality creation of new knowledge? These questions will be at the core of a newly established European Union expert group: The Expert Group on Indicators for Researcher's Engagement with Open Science and its Impacts. The group will synthesize existing research on open science and scholarship and translate this into policy recommendations. The group will deliver its report on these problems at the end of 2018, and will base its recommendations on the consultation of relevant stakeholders. The advice will build further on the recommendations of the EU Open Science Policy Platform, the EU Expert Group on Altmetrics, and on the reports of the EU Working Groups on Skills and Rewards.

Open research and scholarship has different meanings in different disciplines in the sciences, social sciences and arts and humanities. Hence, the new Expert Group will develop recommendations that can be adapted to the specific needs of a particular field of research. In addition, national and regional contexts may also influence the priorities that should be set regarding open research and innovation practices. The Expert Group approaches opening up knowledge creation and circulation as a truly global challenge with different requirements in different regions. The Expert Group will pay specific attention to the current use of indicators in research assessment and their implications for opening up current knowledge producing and sharing practices.

The Expert Group is chaired by Paul Wouters, Professor of Scientometrics, Leiden University. Other members are Merle Jacob, Professor of Business Administration, Lund University; J. Britt Holbrook, Assistant Professor of Philosophy, New Jersey Institute of Technology; Lynn Kamerlin, Professor of Structural Biology, Uppsala University; Alis Oancea, Professor of Philosophy of Education and Research Policy, University of Oxford; and Ismael Ràfols, Research Fellow at Ingenio (CSIC-UPV), Universitat Politècnica de València.

For inquiries about the work of the Expert Group, please contact the chair Paul Wouters.

Members of the Expert Group

Paul Wouters is professor of scientometrics and director of the Centre for Science and Technology Studies at Leiden University. He has published on the history of the Science Citation Index, on and in scientometrics, and on the way the criteria of scientific quality and relevance have been changed by the use of performance indicators. He has also studied the role of information and information technologies in the creation of new scientific and scholarly knowledge. Paul is Chair of the board of the Dutch Graduate School Science, Technology and Modern Culture. He is a member of the editorial board of Social Studies of Science, Journal of the Association of Information Science and Technology, and Cybermetrics, and sits on various advisory boards of international programs and projects. Currently, he is involved in, among others, the PRINTEGER project on integrity in science, the Center for Research Quality and Policy Impact Studies at NIFU in Oslo, and he is member of the program board of the ZonMW program to promote responsible research behaviour. ORCID

J. Britt Holbrook is a philosopher who earned his PhD from Emory University (2004). His postdoctoral research, with Robert Frodeman at the University of North Texas and with the support of several grants from the National Science Foundation, explored the use of broader societal impacts criteria in the peer review of grant proposals, open access, and the development of quantitative metrics of broader impacts. From 2008 – 2014, Holbrook served as Assistant Director of UNT’s Center for the Study of Interdisciplinarity. After two years (2013 – 2015) as Visiting Assistant Professor in the School of Public Policy at Georgia Tech, Holbrook joined the Department of Humanities at New Jersey Institute of Technology, where he is currently Assistant Professor. In addition to his work on science and technology policy, Holbrook also conducts research on the ethics of science and technology. As a member of the American Association for the Advancement of Science’s Committee on Scientific Freedom and Responsibility (2012 – 2018), Holbrook was one of the co-authors of the AAAS Statement on Scientific Freedom and Responsibility. ORCID

Merle Jacob is Professor in Research Policy at Lund University and the UNESCO Chair in Research Management and Innovation Systems. Merle is a member of the Board of the Swedish Foundation for Humanities and Social Sciences. She has previously been Director of the Research Policy Institute, Lund University and the Centre for Technology, Innovation and Culture and the University of Oslo (2006-2010). Professor Jacob’s current research focuses on the role of research funding instruments in changing and institutionalizing new norms and rules of conduct in research. ORCID

Lynn Kamerlin is a computational biologist who obtained her PhD in theoretical organic chemistry under the supervision of Dr.John Wilkie (2005) at the University of Birmingham, after which she was a postdoctoral researcher in the labs of Profs. Stefan Boresch at the University of Vienna and Arieh Warshel at the University of Southern California. She joined the faculty of Uppsala University in 2011, where she is currently a Full Professor of Structural Biology. She has published almost 80 publications in leading journals, and has received numerous prestigious grants and distinctions for her work, including an ERC Starting Grant, a Wallenberg Academy Fellowship, and a Human Frontier Science Program project grant. In addition to her research interests, she is actively engaged in science policy, and is one of the co-authors of the Bratislava Declaration of Young Researchers, as well as having been the co-chair of the SGHRM working group on education and skills, with a focus on the implementation of open science skills training. Finally, she was the Chair of the Young Academy of Europe (YAE) in 2014-2015. ORCID

Alis Oancea is Professor of Philosophy of Education and Research Policy at the University of Oxford, where she is also Director of Research in the Department of Education and Special Advisor on research impact. She has published on research policy, governance and assessment, with particular interests in conceptions and indicators of research quality, impact and knowledge exchange across sectors, disciplines and modes of research, the influence of assessment on individual researchers and their institutions, cultural value from research, performance-based research funding systems, ethical research, and professional knowledge. In her writing, Alis has challenged divisive interpretations of research methodologies and the use of conceptually underdeveloped and ethically unconsidered metrics for research impact and quality in the governance of research and higher education. She has extensive panel and rapporteur experience for grants and publications nationally and internationally, has been an advisory board member for policy initiatives and research programmes, sits on several editorial boards, and has held elected offices in learned societies. She co-edits the Oxford Review of Education. ORCID.

Ismael Rafols is a research fellow at Ingenio (CSIC-UPV, Universitat Politènica de València). Ismael works on research assessment methods, in particular on plural S&T indicators and visualisations for informing evaluation, funding and research strategies. His current focus is on the use of research portfolios as a means of interrogating research orientation. In particular, he is developing mapping methods to explore the cognitive landscapes for societal problems such as bird flu or obesity so as to facilitate deliberation of diverse stakeholders on research prioritization. He received a PhD in biophysics from Tohoku University (Sendai, Japan) and was a postdoctoral researcher in nanobiotechnology at Cornell University before moving to SPRU (University Sussex) in 2004, where he worked until 2012. ORCID

Final Report of the Expert Group on Altmetrics

The Expert Group on Altmetrics outlines in this report how to advance a next-generation metrics in the context of Open Science and delivers an advice corresponding to the following policy lines of the Open Science Agenda: Fostering Open Science, Removing barriers to Open Science, Developing research infrastructures and Embed Open Science in society.

The report will be presented and discussed at the Open Science Policy Platform on 20 March 2017

The report can be downloaded here  PDF icon 796 KB

DG Research and Innovation has established an Expert Group on Altmetrics which will conduct its work over the whole of 2016.

The Expert Group will, among other:

  • Categorise and review different altmetrics and their relationship to more established scientometrics
  • Define the features of a 'responsible metrics' aimed at a responsible use of altmetrics to advance open science, able to track desirable impacts, and qualities of scientific research
  • Develop an agenda for the development of such a 'responsible metrics'

The Expert Group on Altmetrics advices DG Research and Innovation. Altmetrics is a main topic of the European Open Science Agenda which will be further developed and implemented with support of the Open Science Policy Platform (see Open Science Policy Platform)

Members of the Expert Group

James Wilsdon, chair

Professor of Research Policy and Director of Impact and Engagement in the Faculty of Social Sciences at the University of Sheffield.

Paul Wouters

Professor in Scientometrics

Robert Frodeman

Professor in philosophy at the University of North Texas

Judit Bar-Ilan

Professor at the Department of Information Science at Bar-Ilan University

Elisabeth Lex

Assistant Professor at Graz University of Technology

Isabella Peters

Professor Web Science at ZBW Leibniz Information Center for Economics and Kiel University