REF cycles ‘force academics to rush out poorer quality research’

An extensive study provides new backing for a claim long advanced by those working in UK universities: that the research excellence framework forces academics to produce scholarship in greater quantity but of poorer quality.

The paper by academics at the London School of Economics, UCL, the University of Oxford, the International University of Catalonia and the Free University of Berlin examines 190,963 publications that were submitted to the last REF and were published between 2008 and 2013, as well as 211,694 submissions to the REF’s predecessor, the research assessment exercise, dating from between 2001 and 2007.

The authors find evidence that academics rushed outputs to print in order to meet the deadline for submission: the number of submissions published in 2007, ahead of the RAE deadline, was 35 per cent higher than the total for 2008, the first year covered by the last REF.

However, outputs published shortly before the deadline appeared to be of poorer quality, the authors say, since they received 12 per cent fewer citations than those published the following year.

In addition, the UK’s share of papers published in low-impact journals – taken as another proxy of quality – increases ahead of a submission deadline, the authors find.

In all cases, the results were more pronounced in disciplines that typically have a slower publication timescale, such as history, compared to those with a quicker turnaround.

The findings emerge as UK universities prepare for the next REF, which will be published in 2021.

Lead author Moqi Groen-Xu, assistant professor of finance at the LSE, said that the findings suggested that there may be merit in adopting different assessment windows for different fields.

“If you work in a fast-paced field such as computer science, an evaluation every five years may not matter so much in terms of which projects you pursue, or which journals you publish in,” she said. “But if your projects can take more than five years, the REF can be really disruptive.”

She told Times Higher Education: “If you give researchers too much time, they operate under less pressure and may slack off, or are reluctant to cut off ambitious projects that have not taken off despite a lot of investment. If you give them too little time, they may stick to the low-hanging fruit and more established research streams, and publish in easier journals. It is unfortunate that designers of across-field evaluations often forget that research areas differ in where the sweet spot is.”

Commenting on the data, Peter Coveney, professor of physical chemistry at UCL, said it was another sign that the UK “has now become enslaved to the process of performance evaluation of academics”.

“Anyone familiar with the frustration of doing novel research knows that results cannot be summoned to order,” he said. “And yet this study shows conclusively that we see many more people entering publications closer to the deadlines for these assessment exercises, with damaging consequences on the quality of their work.”

rachael.pells@timeshighereducation.com