Researchers ‘unaware’ of extent of assessment influence

Academics are unaware that research assessment fundamentally shapes their work practices, even though they consciously endeavour to turn assessment exercises to their strategic advantage.

A study has uncovered ambiguous attitudes to research assessment exercises, with some academics – particularly those in precarious employment – intensely aware of them, while others are all but oblivious.

Many researchers treat research assessment as a game of “hunting for points” and make minimal efforts to “comply on the side” without allowing the assessment process to derail their research focus, the study found.

Despite this, the process has a dominating influence not only on where and what researchers publish but also the nature of their collaborations, fieldwork, peer reviewing and journal editing and even the types of research they consider possible, found the paper, published in the journal Studies in Higher Education, 

The study investigated Danish academics’ attitudes to their country’s national research assessment exercise, the Bibliometric Research Indicator (BFI). It found that the BFI guided considerations well beyond which journals researchers should target.

“They felt under pressure to produce publications quickly in high-quality international journals,” said Julie Rowlands, an associate professor in education leadership at Deakin University, who co-authored the paper with Susan Wright from Aarhus University in Denmark. “That influenced the types of projects they chose, how they pursued them, how they framed them – fundamental decisions about the nature of the research, not just what they published out of it.

“And they weren’t aware of that until we pointed it out, based on some of the responses they had made.”

The research cites the example of a postdoctoral researcher who insisted that the BFI did not affect how she approached her research.

Later in the same interview she related how she had sent PhD students the “BFI lists” of journals that attracted assessment points, and encouraged them to target the high-scoring publications. When reminded that she had earlier claimed that the BFI did not affect her work, she conceded that it exerted more influence than she had realised. “It kind of infiltrates your brain,” she said.

Dr Rowlands speculated that academics regarded themselves as inhabiting “parallel universes”. One revolved around disciplines while the other involved employment obligations to universities.

“[They] recognised that they were in those two spaces, but they didn’t see them as connected,” she said. “We found that what happened in one universe did affect the other.”

The research involved interviews with academics in the natural sciences and humanities departments of an unspecified Danish university. Dr Rowlands said the paper was based on “a small case study” and made no claims over whether other countries’ assessment exercises would influence researchers in similar ways.

But prior research suggested that they probably did, particularly in nations with metrics-based assessment systems such as Australia and New Zealand.

Dr Rowlands added that the UK’s research excellence framework was unlikely to have such a marked influence because it was not a bibliometric system. Assessments of UK researchers were based on judgements of their work by panels of disciplinary experts, rather than rankings of the journals in which they had published.

“But the academics are still being measured, monitored and reported,” she noted, adding that “the very act of measuring and monitoring” influenced people’s conceptions of merit. “There is still very much the potential for [this] process to affect the nature of research practice,” she said.

john.ross@timeshighereducation.com