More than a game of numbers

The fever-pitch atmosphere in universities as the deadline for submissions to the research excellence framework looms is not unlike that in football clubs engaged in end-of-season promotion or relegation scraps.

Nor are universities any less prone than football clubs to stretching the rules to gain an advantage. The usual gripes abound about universities poaching each other’s star researchers, recruiting temporary staff on fractional contracts and submitting only a small portion of researchers. The REF’s impact element also permits the game-players to base the number of researchers submitted on the number of solid impact case studies they can detail – rather than, as was intended, the other way around.

Another major concern about the REF revolves around the treatment of staff who are not submitted. Non-submission has always incurred stigma, but there is a sense that the consequences are especially harsh this time. Earlier this month, we highlighted a survey by the University and College Union that suggested that high percentages of staff in some institutions have been told that those not submitted will be demoted, disciplined or even dismissed. Despite the small sample sizes and the possibility that institutional messages may have been misinterpreted, it is hard to believe there can be so much smoke without any fire.

As in football, university managers’ jobs are tied to their ability to secure ever higher league table positions

How did it come to this? Our cover feature this week notes that the first research assessment exercise in 1986 asked departments to submit just five outputs and four pages of general description.

The architect of that system, Peter Swinnerton-Dyer, former vice-chancellor of the University of Cambridge, believes this was adequate to get the pecking order “roughly right”. But universities lobbied to be allowed to submit more, and the threat of lawsuits from those who lost out led to the elaborate rules that now generate so much bureaucracy and facilitate gaming.

Rama Thirunamachandran, the architect of the 2008 RAE and now vice-chancellor of Canterbury Christ Church University, says the impact element of the REF was the price to pay for quelling the Treasury’s demand for a less expensive metrics-driven alternative. Metrics have well-known shortcomings but they are also cheap, transparent and, arguably, sufficient to get things “roughly right” at a departmental level. The case for their use in the next REF – at least in the sciences – will surely be pressed hard.

But they would not end game-playing. Nor would they convince institutions to go easier on those who might just have been unlucky in their recent research. Mitigating the worst aspects of the REF will also require universities to conduct themselves more humanely and realistically.

As in football, managers’ jobs increasingly depend on their ability to secure ever higher league table positions. This drives them to put ever more pressure on their team to achieve that (the difference being that researchers – well, most of them – are not overpaid prima donnas who can tell the manager to get stuffed).

It is natural for universities and football clubs to strive to do as well as they can. But a dispassionate perspective reveals that most fluctuate within only a relatively narrow range of positions, and that for every one that rises, another must fall. Those Bill Shanklys of the academy who regard the REF as much more important than a matter of life and death need to keep this in mind when deciding how to respond to a disappointing result.

john.gill@tsleducation.com