Evaluating Research: Assessment and Improvement of Current Practices

Project Status
Ongoing
November 15, 2017
At-a-Glance
The IAPartnership will undertake an evidence-based study of current research evaluation practices across the sciences. It is one of the three projects identified by the IAPartnership for implementation starting in 2015. As the global research enterprise has grown, so too has the demand for more formal judgements with which to evaluate research. The current system of scientific publishing and advances in information technology has led to sophisticated techniques and metrics to assess the quality of both journal articles and researchers. In recent years, however, criticisms of current research evaluation practices have emerged. Critics have found that journal-based metrics may create perverse incentives that lead researchers to cut corners or use questionable methods that undermine the advancement of science, as well as leading to distortions, eg. in career and promotion opportunities.
Key Contacts
  • Tom Arrison
    Executive Director
    InterAcademy Council (IAC)

The IAPartnership will undertake an evidence-based study of current research evaluation practices across the sciences. It is one of the three projects identified by the IAPartnership for implementation starting in 2015.

As the global research enterprise has grown, so too has the demand for more formal judgements with which to evaluate research. The current system of scientific publishing and advances in information technology has led to sophisticated techniques and metrics to assess the quality of both journal articles and researchers. In recent years, however, criticisms of current research evaluation practices have emerged. Critics have found that journal-based metrics may create perverse incentives that lead researchers to cut corners or use questionable methods that undermine the advancement of science, as well as leading to distortions, eg. in career and promotion opportunities.

The challenges posed by current research evaluation practices and the possible negative effects on the global research enterprise, particularly through the use of journal-based metrics, requires a focused response by the world’s scientific leadership, as represented by the world’s academies of science and medicine. The IAPartnership proposes an evidence-based study that will describe current research evaluation practices, how these practices affect research being performed and whether they encourage adverse trends such as perverse incentives, and finally, propose feasible alternatives or actions that can be taken to improve research and hence science-career evaluation practices.

This information will be gathered during a series of regional workshops, using the IAPartnership’s ability to convene international experts and collate experiences from around the world.

The two-year project will result in a final report that will summarize the current system and its issues. It will present recommendations to various stakeholders to ensure that research evaluation practices do not undermine research but rather serve science by advancing quality and integrity. While seed funding for the project has been committed by the German National Academy of Sciences, Leopoldina, efforts are currently underway to secure additional resources for this study.