REF 2021: Leaders say increased impact weighting ‘justified’

UK research leaders have defended the increased weighting of impact to decide scores in the Research Excellence Framework amid concerns that it unfairly favours scientific subjects and research-intensive universities.

In a webinar organised by Times Higher Education to analyse the results of the 2021 REF, David Sweeney, executive chair of Research England, which runs the UK-wide audit of research, said he was pleased that researchers from all disciplines were focused on demonstrating the social and economic contributions of their work.

“I want the UK to say that every discipline is focused, to a considerable part, on research activity and sharing the fruits of it with partners [in order to] help every citizen of our country,” said Mr Sweeney. “I think we can do that.”

Mr Sweeney criticised the “crude metrics” used by other countries to assess impact – an approach that worked for only a “few disciplines”, he said. The UK’s narrative approach to impact, via case studies, delivered a “richness” to assessment not found elsewhere, he continued.

His comments come after the results of the latest REF show that medical institutions excelled on impact, which accounted for 25 per cent of scores, up from 20 per cent in 2014. Arts, humanities and social science disciplines scored less highly on research impact, while impact scores tracked closely to the ratings of outputs – outcomes that helped scientific subjects, as well as older, more science-intensive universities, to score more highly overall.

Colette Fagan, vice-president for research at the University of Manchester, told the event that she believed there was a “bit of room for moderation” to ensure that the impact of research in the arts, humanities and social sciences was rated more in line with science, technology, engineering and mathematics (STEM) research.

However, Professor Fagan said, it was a myth that it was impossible to demonstrate the impact of non-STEM research, even if it was “less linear” than the typical “invention to market” impact often found for life science research.

“When I started at Manchester in 2014…I thought it would easier to evidence the impact in panel A [medicine, health and life sciences] and B [physical sciences, engineering and mathematics] subjects, but it was a different kind of impact,” she said.

“I am less concerned about panel A is higher than panel B for impact – I’m more interested in looking at the impact cases and stories,” which, she believed, were the “most exciting” results of the exercise.

Asked whether the impact scores merely repeated the score for outputs – given their close correlation – Professor Fagan said she would “expect good research to produce good impact”, adding that “one would expect some correlation as we are talking about research”.

However, impact had “shifted behaviour” and asked researchers to “think what their research ambition and agenda is, and what is the range of aims that a project will deliver”, she said.

This year’s REF has introduced a number of changes, including a requirement for all research-active staff to enter at least one output into the exercise, whereas the 2014 exercise allowed institutions to select researchers, who were required to submit at least four outputs.

Asked if the 2021 REF had been less onerous than the 2014 exercise, Professor Fagan said “some elements have been easier” but overall, it had not been any less time-consuming. “We have shifted from looking at individuals to selecting based on the portfolio of a unit – it is worth doing, but it has not shifted the burden,” she added.

In future exercises, Research England should consider changing the requirements for the environment aspect of the REF – worth 15 per cent of assessment – in which units of assessment were asked to write narratives about how they support a positive research culture. “The environment statement can be very time-consuming [considering] the proportion of weighting that it gets,” Professor Fagan said.

jack.grove@timeshighereducation.com