When the peer-reviewed Open Information Science Journal accepted a computer-generated hoax paper from the “Centre for Research in Applied Phrenology” (CRAP) last summer, it became all too clear that peer review is no guarantee of academic rigour. But this was far from the only controversy involving peer review in recent years. From the full retraction of Andrew Wakefield’s paper suggesting a connection between autism and the combined measles, mumps and rubella vaccine, through the “Climategate” scandal, to the 14 stem-cell biologists who complained that an obstructive clique was blocking the publication of quality research, the methodology has been under constant attack.
It will hardly come as news to anyone who works in the academy that peer review is imperfect, open to manipulation and can be compromised by conflicting interests. But as the stories mentioned above show, debates about peer review are no longer confined to academia, but have become a matter of considerable public interest. This is in no small part because peer review is now the official process of approval needed for research findings to become the basis of policy.
“Evidence-based policy” is a concept that was popularised under the New Labour government, and is now deployed by both the present coalition government and its most outspoken critics, the latter using it as a stick with which to beat the former. Imran Khan, director of the Campaign for Science and Engineering, joked recently: “It would be great if we could engineer an evidence-based policy generator for government.”
Basing policy on evidence is widely believed to be a good thing, because it appears to place political decisions on a more scientific footing. But the currency of scientific fact is not the same as the currency of political meaning. Conversion from one to the other requires a process of interpretation and argument. If this intermediate process does not occur, research findings are treated as though political meaning comes pre-packaged with them. This gives the political prejudices of the day the undeserved imprimatur of researchers’ academic credentials.
Making evidence the primary basis for policy seems to indicate that politicians are taking academics seriously. In fact, this is a swindle. The politicians are swiping the academics’ legitimacy, while the academics are left making political and moral calculations. The academics in question may feel flattered, but their integrity and independence of thought has been compromised, because their pursuit of knowledge is now inextricably linked with political ends rather than being an end in itself.
This helps to provide a context for one of the most infamous emails that emerged from the Climategate affair. UK climatologist Phil Jones told his US colleague Michael Mann: “I can’t see either of these papers being in the next IPCC report” – a reference to one of the Intergovernmental Panel on Climate Change’s reports collating research and steering policy. “Kevin and I will keep them out somehow – even if we have to redefine what the peer review literature is.”
The political stakes involved in the work these scientists do are so high that it makes the scientists fearful of peer review that allows the “wrong” research to be taken into account.
Once politicised in this way, peer review becomes a sort of quarantine, where ideas can be contested only by a select few before being presented to us as a fait accompli. Take, for example, the attitude of epidemiologists Kate Pickett and Richard Wilkinson, authors of the book The Spirit Level: Why More Equal Societies Almost Always Do Better. They state imperiously: “Almost all of the research presented and synthesised in The Spirit Level had previously been peer-reviewed…In order to distinguish between well-founded criticism and unsubstantiated claims made for political purposes, all future debate should take place in peer-reviewed publications.”
Such disdain for arguments motivated by “political purposes” speaks (non-peer-reviewed) volumes.
While academics disparage politics, politicians seem increasingly to believe that their job consists of taking ready-made academic ideas and plugging them in to the framework of evidence-based policy. The vision of an “evidence-based policy generator” conjured up by Imran Khan illustrates how little room this leaves for conscious political and moral agency.
Peer review is valuable and worth defending, but only inasmuch as it promotes impartiality and standards within specialist fields. It should not be used as an arbiter of how we run our affairs. We already have a system for that – it’s called politics. And we are all qualified to participate in it, by virtue of being born human.
Sandy Starr is communications officer at the Progress Educational Trust. He is producing the Times Higher Education-sponsored debate, “End-of-the-Peer Revue: Has the Peer Review Process Lost Credibility?” at the Battle of Ideas festival on 31 October.