Library Element Report

Towards a reform of the research assessment system

👍
0
Uploaded by RRI Tools on 30 November 2021
Last modified on 30 November 2021

Towards a reform of the research assessment system: scoping report. European Commission. Directorate-General for Research and Innovation. November 2021.

Reforming research assessment is increasingly considered a priority to ensure the quality, performance and impact of research. Reform, however, requires cultural and systemic changes which are proving to be very complex and slow to implement. During the period March-November 2021, the European Commission consulted European stakeholders on how to facilitate and speed up changes. This scoping report presents the findings from the consultation, identifies the goals that should be pursued with a reform of research assessment, and proposes a coordinated approach based on principles and actions that could be agreed upon by a coalition of research funding and research performing organisations committed to implement changes.

Table of Contents

  • EXECUTIVE SUMMARY
  • INTRODUCTION 
  • RATIONALE FOR REFORMING RESEARCH ASSESSMENT
  • EU POLICY AND POLITICAL CONTEXT
  • PROPOSED APPROACH
  • PRINCIPLES FOR A REFORMED RESEARCH ASSESSMENT SYSTEM 
  • ACTIONS THAT SIGNATORIES OF AN AGREEMENT COULD COMMIT TO
  • ORGANISATION AND MONITORING
  • ANNEX 1: CONSULTATION OF STAKEHOLDERS
    • 1 Description of the consultation process
    • 2 Consultation meetings with stakeholders 
      • 2.1 Meeting of 18 March 2021: List of organisations having participated
      • 2.2 List of bilateral meetings with stakeholder organisations 
      • 2.3 Other meetings, debates and consultations 
  • ANNEX 2: LITERATURE CONSULTED
  • ANNEX 3: A FEW DEFINITIONS 

 

EXECUTIVE SUMMARY

Assessment of research quality and impact, and of researchers’ performance, is fundamental to selecting research proposals to fund, to deciding which researchers to recruit, promote or reward, and to identifying which research units and institutions to support.

The research process is undergoing digital transformation, and is becoming less linear and more collaborative and open, and more multidisciplinary with a larger diversity of outputs. At the same time, the current research assessment system often uses inappropriate and narrow methods to assess the quality, performance and impact of research and researchers. Notably, the quantity of publications in journals with high Journal Impact Factor and citations are currently the dominant proxies for quality, performance and impact. Many research funding and performing organisations are already taking steps to reform and improve the way they assess research and researchers, but progress remains slow, uneven and fragmented across Europe.

From March to November 2021, the European Commission consulted European and international stakeholders on how to facilitate and speed up reform so that the quality, performance and impact of research and researchers are assessed on the basis of more appropriate criteria and processes. The consultation identified objectives and outlines of a reformed research assessment system, with principles and actions that could be agreed between research funding and research performing organisations, as they have the responsibility to define their criteria and processes to assess their researchers and research projects.

The proposed way forward consists of a European agreement that would be signed by individual research funding organisations, research performing organisations and national/regional assessment authorities and agencies, as well as by their associations, all willing to reform the current research assessment system. The aim is for research and researchers to be evaluated based on their intrinsic merits and performance rather than on the number of publications and where these are published, promoting qualitative judgement with peer-review, supported by a more responsible use of quantitative indicators. The way in which the system is reformed should be appropriate for each type of assessment: research projects, researchers, research units, and research institutions. A reformed system should also be sufficiently flexible to accommodate the diversity of countries, disciplines, research cultures, research maturity levels, the specific missions of institutions, and career paths.

The agreement would confirm the commitment of the signatories to changes, along commonly agreed objectives, principles and actions. It would offer a space for individual institutions to test changes, for mutual learning, and to more safely and efficiently engage in reforms. An implementation plan would be established by the signatories, including deliverables, milestones and timeframes, in order to translate the commitments into effective changes. Measures for monitoring the progress made and for exchanging information would also be agreed among the signatories to ensure that commitments translate into tangible changes, and to ensure mutual learning for evidence-based changes. Researchers would need to be closely associated to the implementation and
monitoring processes.

 

Open
English

Related Resources