RAE goes Forth and multiplies

A consultation paper on the future of the Research Assessment Exercise has been issued. Bahram Bekhradnia describes its purpose.

The research assessment exercise has some of the characteristics of the Forth Bridge, and we have begun the repainting process less than a year after the conclusion of the last exercise. This week, the four funding bodies involved in the exercise have issued a consultation paper on its future. This provides an opportunity to look not just at its processes, but more fundamentally at its purpose.

The RAE has its critics - there is no shortage of those. However, they need to bear in mind that the primary purpose of the exercise is to produce judgements of quality so that the limited amount of money provided by the government for research can be allocated selectively. There is an enhancement function, too - there is ample evidence that the quality and management of research has improved and that the RAE has played some part in this - but the RAE has not been designed with that in mind.

Given our policy of allocating research funds selectively - a policy which I am convinced is in the national interest - the key question which must be answered is what instrument should be used for making selective judgements. That is why, although the consultation paper asks whether the RAE should be discontinued, it does so in the context that some basis is needed for deciding on relative quality. The alternative would be to spread the money for research thinly, without regard to quality. On the other hand, if a better instrument were available, generally acceptable to the academic community, then we would use it.

However, recognition of the primary purpose of the RAE is not to deny that it now has a much wider role. For example, other funding bodies make use of RAE judgements in their funding decisions; universities and colleges use RAE results in their internal allocations; and research students use them in deciding where to study. In reviewing the exercise we ought to consider whether it could be a more useful tool for others. Perhaps, if it were modified, even slightly, or if different data were collected - or a little more data - then considerable value might be added.

For example, it is sometimes suggested that the exercise should be more formative and developmental, which would require more systematic feedback. In its current form this would undoubtedly make the task of the panels more complicated, and would make the exercise longer and more costly. The increased value might outweigh the cost, but we should not lose sight of the primary purpose. We will continue to need a system that enables judgements to be made about quality, so that research funds can be allocated selectively.

Although we may consider extension of the exercise, its process and cost should be proportionate to the results and to the benefits which it brings. On the basis of returns from institutions themselves, we estimate the cost of the 1996 exercise to have been about Pounds million (including the cost of the time devoted to preparation by academic staff). This expenditure should be set alongside the fact that the results will be used to allocate over Pounds 3.5 billion of funds in the coming years - a transaction cost of three-quarters of 1 per cent, which is considerably lower than processes used by other education and research funding bodies. On the other hand, viewed at the institution level, there is only relatively small movement in the standing of universities and colleges relative to each other and in their consequent funding. Does this suggest that the RAE is being used to crack only a small nut?

The consultation paper asks whether it would be possible to introduce only a partial exercise aimed to catch changes. I suspect that, desirable though this may be, there is no satisfactory process to achieve it. It also asks whether the period between exercises should be longer, whether it should be phased, and whether different subjects should be treated differently. When we asked these questions previously there was a strong feeling among the lower rated departments that the interval should be shorter, and among the higher rated that it should be longer.

The consultation paper invites comments on a number of criticisms levelled against the RAE, and also invites suggestions about how these might be overcome. Among the more worrying criticisms are the suggestions that the exercise encourages academics to pursue "orthodox" research strategies which they believe may be favoured by panels; and also that it encourages them to neglect non-research activities and devote themselves increasingly to research, because that is where the greatest rewards are. Other issues which we must address include the need to keep the exercise responsive to changes in academic practice; to ensure a greater international dimension; and to ensure greater compatibility of practice and standards between panels.

We have to beware of attributing all the ills of academic life to the RAE. Pleasing the panel is a problem with any sort of peer review, and the predominance of research in academic life is a worldwide phenomenon. Rather than attack the RAE we need to address such problems in a positive way, for example, by providing incentives for teaching to match the incentives provided for research, and we are considering how to do this.

The consultation just beginning will be open-ended and comprehensive. As well as formal responses from institutions, we are enabling direct contributions via an electronic discussion group on the Internet. One of the strengths of the RAE has always been that it has been open to modification in response to the views of the academic community. And if out of the consultation comes a consensus for a different approach to the identification and encouragement of high quality, then we should be prepared to embrace this.

Bahram Bekhradnia is director ofpolicy at the Higher Education Funding Council for England.