Official estimates of the cost of research assessment to universities are “disingenuous” and the real bill could exceed £1 billion, a senior academic has estimated.
An independent report by PA Consulting puts the total cost to higher education institutions of the 2008 research assessment exercise at £47 million, amounting to an average of £613,000 for each institution, or £88,000 a year over seven years.
But according to Robert Bowman, director of the Centre for Nanostructured Media at Queen’s University Belfast, that figure amounted to just one full-time senior-level salary a year, which was “not even at the races” for an institution such as his.
Based on conversations with colleagues across the sector, Professor Bowman has produced a “guesstimate” of the real cost of the Download a full spreadsheet of Robert Bowman’s calculations
Hit and miss metrics: ‘Throw of dice would give more accurate REF prediction’
An examination of university departments’ “h-indices” failed to accurately predict the research excellence framework results, a study has concluded.
Prior to the publication of the results in December, a team of physicists published their predictions in four subjects – physics, chemistry, biology and sociology – based on observation of departments’ h-indices during the assessment period.
Broadly, the h-index measures the number of papers that garnered a significant number of citations. Although the physicists were opponents of metrics, they wanted to examine, in a “neutral” way, whether a department’s h-index correlated with its REF score. Some advocates of replacing peer review with metrics, as a way to save universities money and effort, have pointed to supposed correlations between h-indices and results of the 2008 research assessment exercise.
The group’s follow-up paper, Predicting Results of the Research Excellence Framework Using Departmental h-index – Revisited, published on the arXiv preprint server, concludes that although correlations exist, they are not nearly strong enough to justify replacing peer review with metrics.
Most worryingly for proponents of metrics, in three of the subjects examined, the h-index correlates more closely with overall REF scores – which include impact and environment elements – than it does with the outputs element in isolation.
They also found that h-indices were unable to predict whether a department would rise or fall in the REF ranking. One of the paper’s authors, Ralph Kenna, reader in mathematical physics at Coventry University, said: “Managers would get more accurate predictions by tossing dice.”
Paul Jump
