Sweeney: REF requires ‘fundamental rethink’

The UK’s Research Excellence Framework (REF) is in need of a “fundamental rethink” to reflect the changing understanding of how research systems work worldwide, according to the outgoing executive chair of Research England.

But David Sweeney, writing in a Higher Education Policy Institute (Hepi) collection of essays on research evaluation, published on 1 September, says his successors face a choice between implementing “modest modifications” in the next few months to allow time to engage with international debates – with a view to making bigger changes in the following cycle – or taking a few years to complete a full redesign.

“My reason for a fundamental rethink is that we are thinking, worldwide, about how we understand the working of the research system, about research culture, incentives, rewards and success criteria,” Dr Sweeney writes.

“The UK should play a full part in that debate, which has some way to go, and, as more settled positions emerge, capture the learning in the way we design our assessment exercise.”

During his term in office, Dr Sweeney oversaw the most recent REF cycle, which resulted in elite institutions claiming a smaller share of quality-related (QR) grant funding and several more modern universities receiving significant boosts to their research budgets.

Nick Hillman, Hepi’s director, noted that as Dr Sweeney is replaced by Dame Jessica Corner at the top of Research England, “a lot of things are up in the air” and in all likelihood the next REF could “look considerably different”.

But he added: “Although people criticise the REF a lot, it actually has worked pretty well; it is something other countries look towards when they are designing their own evaluation mechanisms.

“Every REF has been at least a little bit different to the previous one. It has been a story of constant improvement rather than starting with a blank sheet of paper. I don’t see any reason to think that will change.”

In his own chapter in the report, Research Evaluation: Past, present and future, Mr Hillman suggests that a broader range of research outputs beyond traditional journal publications should be given greater weighting in the next REF, while the link between access to QR funding and institutions’ undergraduate fee status should be severed.

In response, Dr Sweeney writes that changing the link between education and research that underpins the funding would be “a matter for legislation, and therefore a national discussion, not a matter for the machinery of government”.

He goes on to state that the international focus on research culture “suggest[s] that there is widespread support for a much stronger emphasis on recognising and rewarding desirable attributes of the research system”.

But it remains a “significant challenge to capture sufficient rigour and richness in the assessment of research environment”, which makes this “the key issue” for future research assessment exercises.

He adds that there needs to be discussion about who is inside “the academy” given that so many technicians, statisticians, librarians and research managers make up modern research teams.

Overall, Mr Hillman said, the REF remains a powerful tool from a policy perspective because it equips the research community with the evidence needed to convince the Treasury to continue funding the sector fairly and because without it, any future science minister could struggle to make their case when competing with other government departments.

tom.williams@timeshighereducation.com