Let's broaden the definition

One of the most frequent criticisms made of the research excellence framework is that it has too narrow a conception of acceptable academic "output". Peer-reviewed academic journal articles are of overwhelming importance, with monographs "scoring" the same as single articles, and some kinds of publication scoring nothing at all.

While it is important to fight the managerialism that the REF represents, the exercise is unlikely to go anywhere anytime soon, so perhaps the time has come to broaden it from within. Of course, the new emphasis on research "impact" was designed in part to do this, but in practice it appears to be leading towards yet another box-ticking exercise in defining the indefinable.

A more productive way of broadening the REF would be to look again at the reason why journal articles have taken on such overwhelming importance - peer review. Despite its flaws, it remains one of the few effective ways in which some kind of standard of scholarship can be affirmed. So if peer review could be spread more widely, maybe it would allow for a wider range of material to become acceptable to the REF.

One idea that has been aired in these pages is to have a voluntary pre-publication peer-review system for monographs. But perhaps a better place to start would be the journals themselves. Most publish a range of material besides articles, including book and essay reviews, responses, round tables and interviews. In reputable journals, the editor will look at everything published, but the full weight of the peer-review system is generally not brought to bear on the periphery surrounding the article core. Consequently, none of this stuff "counts" in REF terms. This is a pity, as it is often in the less formal pieces that cutting-edge thinking is done. It is this "Cinderella" content that keeps journals journals, rather than just collections of articles.

In these days of online access, it is becoming increasingly difficult to justify the journal as a coherent publication. A revalorisation of a wider spread of journal content could reaffirm the publications' value as literature, worth reading from cover to cover. If journals could be treated as wholes, then some of the atomisation of the academy - in which isolated scholars look for isolated articles in their narrow areas of interest - might be resisted.

How might peer review of a wider spread of journal writing work? An essay review or short response piece wouldn't need the same rigorous assessment that articles get, but they should at least be read by more than one person able to offer comments and suggest improvements. Further, opening up journals to comment threads, common in other online publications, might also offer new possibilities for scholarly review and criticism. Contributors could even be encouraged to redraft published pieces in light of the comments.

Certainly there are difficult issues regarding the unpaid time that peer review draws on, although reducing the number of journal articles required for REF submission might reduce the glut of proposals.

What is most important initially is spreading the idea that a broader range of journal content should be taken more seriously by the bodies that police and allocate funding to the academy. Essays, reviews and the like should probably not count for as much as articles, but they should at least be given appropriate credit when assessing scholars' productivity and excellence.

This kind of change could only come about if there were some kind of grass-roots pressure from academics themselves. If individual journals were to take a positive decision to revalorise their non-article content - to think of themselves as publications with a certain aesthetic - then the means to do so might follow.

The REF process is one contributing factor to the simultaneous mushrooming of the number of journals and the erosion of their literary qualities. Extending peer review to all aspects of journal output would not only offer hope that the REF's stranglehold on academic publication might be loosened, it could also breathe new life into the stale conventions of academic publishing.