‘Low’ UK take-up of responsible metrics agenda causes concern

The “low take-up” of a declaration on responsible metrics in the UK research community suggests that a performance measurement culture is still strong within the country’s academy, a sector leader has warned.

David Sweeney, director of research and knowledge exchange at the Higher Education Funding Council for England, suggested that the “question of has the tide turned” when it comes to the use of metrics loomed large within academia, but he was hesitant to be overly optimistic.

“I think the tide has turned at a certain level of discussion…but as we’ve seen with the low take-up of the [San Francisco] Declaration on Research Assessment, I don’t believe the tide has turned in the research community as a whole or, indeed, necessarily in universities, who face very difficult problems about their own assessment frameworks,” he said.

“I think we’ve got to recognise that and engage in much more advocacy.”

The San Francisco declaration, also known as Dora, calls on the academic community to stop using journal impact factors as a proxy for the quality of scholarship. On 8 February it was announced that all seven of the UK’s research councils had signed the declaration, which was initiated at the annual meeting of the American Society for Cell Biology in 2012.

Mr Sweeney’s comments, at a conference hosted by the Forum for Responsible Research Metrics, were reflected in the results of a new survey of UK research organisations about the metrics culture within their institution, which were presented at the event.

More than three-quarters (78 per cent) of the 96 respondents, most of which were higher education providers, said that their organisation did not have a research metrics policy, while the same share said that their organisation had not signed Dora.

While almost a third (32 per cent) of survey respondents said that they were considering signing Dora, the same share said that they were not considering doing so. A further 13 per cent of participants said that they had considered signing the declaration but had decided not to proceed.

When asked what action had been taken at their organisation to promote the principles of Dora or similar initiatives such as the Leiden Manifesto for Research Metrics or The Metric Tide report, more than a third (34 per cent) of respondents gave no answer or an answer that did not respond to the question.

Paul Ayris, pro vice-provost for library services at UCL, who presented the survey results, said that the “evasion” was “a bit worrying” because it showed that there was “a lack of engagement or a lack of confidence in how to take these issues forward”.

Almost two-thirds of participants (61 per cent) said that they saw a value in the UK developing an agreement similar to Dora.

But Mr Sweeney warned against the idea of the UK attempting “to produce something that is very special to the UK”, adding that it must be a “continuing global effort”. He added that academics “who care about” responsible metrics must “lead the advocacy first”.

Lizzie Gadd, research policy manager (publications) at Loughborough University, who also spoke at the event, argued that “responsible peer review is just as important as responsible metrics but the frameworks don’t address this”.

She said that frameworks “need to apply to our people”. “We need robust, humble, transparent, reflexive, diverse people doing metrics. Unfortunately we often have unqualified and overconfident people both generating and interpreting metrics for research evaluations,” she said.

ellie.bothwell@timeshighereducation.com