Let's repair the damage

As the dust settles on the final research assessment exercise, now is a good time to reflect on what it achieved and what may follow. Was it worth it? Will its replacement be better or worse?

The real, although undeclared, purpose of the RAE was to place the burden of rationalising university departments on institutions rather than on central authorities. In particular, there were concerns about the proliferation of poorly resourced centres in costly subjects such as applied science and technology. But once it became clear that there were "extra" resources for those subjects, the exercise was extended to cover research in all disciplines.

This was first done in the context of a much smaller and less diverse university system. The first RAE was held in 1986, and it quickly became clear that it served a number of purposes.

It produced a measure of rationalisation. It enabled spending departments to show the Treasury that research money was being put to good use. It met the need, in an increasingly marketised system, to create a clear pecking order of departments and institutions. And it almost certainly raised the quality of academic research. But all this was achieved at a huge cost.

Ironically, there is not a huge amount of research literature on the RAE. But what seems clear is that its value for money declined over time, as an ever more resource-intensive process led to ever more marginal outcomes in terms of funding transfers between institutions.

The RAE created or reinforced a conservative bias towards discovery research on a conventional disciplinary basis, at the expense of applied or user-focused research across disciplines. Its focus on research has damaged other university activities, not only teaching, but also other forms of scholarship, such as writing textbooks.

It has distorted other activities, too, as better-resourced institutions have bought research "stars" to boost their attractiveness to sponsors, staff and students. Above all, it created or reinforced a degree of hierarchy that is not in the best interests of either higher education or the UK. Ultimately, the RAE was about the allocation of status to institutions, subjects, academic activities, departments and academics.

Its successor, the research excellence framework, will replace or supplement peer review with the evaluation of quantitative data, such as citations. This follows the unheralded intervention in research policy by Gordon Brown, then Chancellor of the Exchequer, in the 2006 Pre-Budget Report. But this was a missed opportunity.

Instead of considering whether a sector- and subject-wide selectivity exercise was still the best use of scarce resources and expertise, or even supported the Government's desire to create a more responsive system, the discussion was quickly reduced to an interminable exercise to determine the correct balance between the various forms of evaluation, at best only limited proxies for research quality. Regrettably, it appears that this agony will continue. What should happen instead?

Clearly, there can be no going back. Selective funding by the research councils and other funders will continue. The real difficulty with the RAE was that it took a narrow view of what is meant by research and then tried to treat all forms of research alike for purposes of evaluation and funding.

A more realistic approach for the REF would be to use a broader definition of research, comprising all forms of systematic inquiry that lead to new information, insights or understanding, and to confine competition for funding to those fields where the research effort is unduly expensive, justifying some degree of concentration on economic grounds.

There could be selectivity where there are concerns about quality, or where it is desired to establish a subject for research purposes. All other forms of research would be funded pro rata to staff effort, with effectiveness and efficiency monitored through an audit process within institutions. Its efficacy could be evaluated by institutional review. It is still not too late to begin to restore some integrity to academic work.