Journals must stop blocking critical comments

Critical comments are of fundamental importance to academic publishing. They do more than peer reviewers do to keep academics from knowingly submitting bad or faked research. And they can be the first step in shifting scientific paradigms.

When I started writing critical comments in 1973, there was a presumption that editors would publish them if possible. They would check them, but people who write comments face a reply from the author, so they check their own work more carefully than any referee or editor ever would. Besides, errors are usually obvious once they have been pointed out.

Comments continue to be read and cited for years. In the week of writing, for instance, more than 200 researchers and students from 12 countries have read 10 of my comments and refutations; seven of these were written more than 20 years ago.

But 15 years ago, the climate in publishing began to change. Some American journals decided that critical comments showed up the fallibility of their vetting process, which could affect the commercial value of their brands. Instead of accepting that all refereeing is fallible, they started running comments past two referees, as well as the editor. Inevitably one of the three would object to something trivial, and this could then be used as an excuse to reject the comment.

A few years ago, I submitted a comment to a journal showing that a paper it had published broke all the rules for surveys. However, one referee disagreed with my statement that it was improper to use members of one organisation to interview members of another when comparing the two organisations (which is just one of the rules). The comment was suppressed, so I put it up on ResearchGate and Academia.edu instead. Now lecturers ask their students to read it before conducting a survey.

More recently, a UK journal developed a cast iron defence against criticisms. When I objected that one of its papers was littered with errors of commission and omission, my comment was considered by the entire editorial board – of 49 people, according to the journal’s website. This level of pre-publication scrutiny is unprecedented.

The 49 – including the editor – found no fault with my economics or facts, but would not even consider publishing my comment because, I was told, the editorial board “expects all comments and rejoinders to be worded in a collegial and constructive manner, tone and style. We feel that yours does fall rather below our usual conventions in this regard.” I did not make any ad hominem attacks or snide remarks – unlike the author of the paper I was criticising – so I asked for clarification. But all I received was a reiteration that I must rewrite my comment “in the sort of tone and style that is normal for contributions to this journal”.

No rational writer would comply with such a hostile demand. I am asked to rewrite my comment to meet secret criteria about toning down criticisms. And if by some miracle I meet these criteria without the criticisms vanishing, the editorial board will send my comment to two referees. Given that the journal’s referees accepted the appalling paper in the first place, how much faith can I have in them?

Even if they pass my comment, the editor may still reject it. Research shows that a significant proportion of editors in the sciences overrule the referees or rewrite their comments. And if the author makes a reply, however wrong, meaningless, or ad hominem, I have no confidence that I will be allowed to write a rejoinder.

I surmise that the journal’s objection is actually to my conclusions, rather than the way I present them; this is not the first time I have seen the words “collegial” and “constructive” used as a demand for omerta.

The journal says that “the priority of the Editorial Board is the journal”. Clearly, we have totally different moral stances. The paper I object to is on the economics of famine. Bad economics kills people – and I am squeamish about killing people. I do not see any moral justification for the continued existence of a journal that refuses to correct falsehoods in any field, but particularly this one. Less seriously, the journal is arguably committing an offence under the UK’s Consumer Protection from Unfair Trading Regulations 2008 by continuing to sell a paper it knows to be full of falsehoods.

It is still possible to submit refutations of entire research programmes to a range of journals. In practice, though, such refutations develop out of a few critical comments on specific papers – and these can only be submitted to the journal that published the paper in the first place.

This means that to avoid the risk of wasting their time, comment writers these days would be wise to seek assurances from the journal about publication before they even begin. The implications for academic integrity – not to mention people’s willingness to attempt to set the record straight in the first place – are obvious.

Peter Bowbrick has retired after a career in economic research and consultancy.