Can peer review survive social science’s paradigm wars?

Peer review is essential to the operation of research communities. But what are the conditions that must be met for it to operate? One is surely that there must be some minimum level of agreement about the task of research and how it should be pursued.

Yet increasingly, in many areas of the social sciences and humanities, there are fundamental divisions not just about the nature of what is being studied and how to understand it, but even about what the product of research should be. For instance: are human beings, organisations and institutions causal agents operating in the world or are they discursive constructions with nothing lying “outside the text”? Is the aim of research to understand the world or to have “impact” on it, including reducing social inequalities?

Involved here are clashes between fundamental commitments, resulting in the “paradigm wars” whose future Nate Gage famously predicted back in 1989. Is peer review compatible with these conflicts? Let me illustrate the problem.

Recently, in reviewing a paper for a journal, I faced a dilemma. The paper made some interesting points, but my view was that it relied on a range of doubtful empirical, theoretical and political assumptions that led to tendentious interpretations of rather thin data. The authors implied that any questioning of these assumptions amounted to an attack on their intellectual and social identities. But I felt that since many readers would not share them, explicit justification ought to be provided.

Given this, I recommended major revisions. The other reviewers were more favourable, though they did identify various issues that needed attention. The editors decided on minor revisions. The resubmitted paper made no substantive changes relevant to my comments. More importantly, the covering letter did not respond to me at all, only to the other two reviewers.

The editors subsequently asked the authors to address my comments, but their response was that since they had a different “onto-epistemological position”, they did not need to offer counter-arguments to the specific points I had made. They claimed that the differences between us were “irreconcilable” and outside the scope of their paper, and that my “standpoint” informed my “personal opinion of ‘valid data’”. They insisted that their “methodologies” had been “thoroughly vetted by the academy as reliable, valid and trustworthy”. In other words, since they were able to appeal to a literature that shared their commitments, there was no need to justify these in the paper.

I disagreed and recommended rejection, especially since, in my view, the paper still had fundamental defects. However, the other reviewer at this second stage recommended publication and commented: “Kudos to these authors for their pushback to Reviewer 1.”

“Pushback” sounds like a military metaphor; the implication seems to be that mere engagement with the critical points I had made would have amounted to surrender in the face of what needed to be repulsed. Note that what is at issue here is not that the authors did not modify their paper; peer review does not require this. The problem is that they at first refused to make any response, and when pressed, simply appealed to their own paradigm commitments. And it worked. The article will soon be published.

Of course, it could be argued that I should have acknowledged the legitimacy of the alternative paradigm to which the authors claimed adherence and accepted that my critical comments were therefore inapplicable, simply reflecting my own paradigm. Many would argue that it is a principle of academic life that diversity in orientation should be welcomed, and I agree up to a point. But there are clearly limits to toleration – indeed, both sides of the dispute discussed here recognised this in practice. The question is: on what grounds should those limits be determined?

The authors were effectively rejecting a key assumption underpinning peer review. They adopted a version of what has come to be called standpoint epistemology, according to which some views are granted credibility – and others denied credibility – on the basis of the social category to which those putting them forward belong. The perspectives of members of marginalised or oppressed groups (or, rather, those who claim to speak on their behalf) are assigned epistemic privilege, while those held by members of what is taken to be the dominant group are rejected, for example as (at best) “opinions”.

So on one side, the limits to what is acceptable were being defined in terms of a paradigm that the authors freely acknowledged is committed to political goals; whereas on my side, the limits were taken to derive from what is required if academic peer review is to operate. The clash arises from the fact that peer review demands that all “peers” be treated as equal, none as epistemically privileged.

Indeed, anonymity is used to render both authors and reviewers blind to (among other things) each other’s social characteristics and political views, as far as possible. All that is held to matter is that they are fellow members of the same research community; and they are required to engage with one another solely on that basis, not according to political or paradigm commitments.

Can abandoning this requirement be tolerated within peer review? It has never been perfect, of course, but if peer review is just another vehicle for paradigm warfare, can it still be justified?

Martyn Hammersley is emeritus professor of educational and social research at the Open University.