Is anonymity or transparency the best solution to bias in peer review?

When reviewing someone’s work, fair judgement is threatened by bias – conscious or otherwise – about their gender, name, nationality, affiliation or career status. That is why a growing chorus in recent years has been calling for new models of peer review to be developed that minimise the opportunity for prejudice to creep in.

Publishers have an obligation to respond to such calls. Interesting experiments have emerged, consisting of varying degrees of openness and achieving varying degrees of success. Fully open peer review, for instance, discloses the identity of both the author and the reviewers to all participants, while collaborative peer review sees referees work together on a single review report or work with authors to improve the paper.

At IOP Publishing, we have introduced two different but complementary approaches to bias reduction at all our self-owned open access journals. As the first physics publisher to adopt these approaches portfolio-wide, we believe the sector will be interested in how they have been received.

THE Campus resources: Understanding peer review – what it is, how it works and why it's important

In the past year, we’ve moved all our own journals over to double-anonymous peer review, whereby the identities of both the reviewer and author are concealed. Our early data suggest that anonymised papers are more likely to be published, and the feedback we’ve received from both sides has been overwhelmingly positive.

It is particularly encouraging to see that enthusiasm for double-blinding is not confined to early career researchers or people worried that their name or location might negatively affect the assessment of their work. Recently, a Nobel laureate anonymised their manuscript, demonstrating a belief in the inherent quality of their research and the publishing system’s ability to recognise it rather than banking on their established reputation.

Some people argue that double-blinding is ineffective because reviewers can still identify authors by looking at their references. However, early indications are that this may not be as big an issue as feared. We’re gathering data on this, which to date show that 85 per cent of reviewers of anonymised manuscripts are not confident that they could accurately guess author identities.

The other method we’ve introduced in our quest to make publication fairer and more robust is transparent peer review. Under this, journal readers are shown the full reviewing history of each paper, including referees’ reports, editors’ decision letters and the authors’ responses. Reviewers may also choose to reveal their identities. We believe that this increases accountability, allows reviewers to be better recognised for their hard work, and can aid the training of aspiring reviewers.

Although several other publishers have adopted elements of transparent peer review, there is great variation in how it is applied. Some publish the reviewer reports but not the correspondence, for instance. And while some journals mandate transparency, others allow authors to opt in.

For now, we’ve chosen to give both authors and reviewers the option to opt in. This will allow us to gauge the true appetite for transparent peer review. Currently, 40 per cent of our reviewers opt to publicly disclose reports and just over half of authors choose to. This figure seems relatively healthy, and is slightly higher than the one Nature reports.

To encourage increased accountability, we are considering automatically enrolling reviewers for transparent peer review, leaving the choice entirely to the authors. However, we’ll trial this first in selected journals to see how it affects reviewer behaviour. You might argue that some reviewers will decline invitations to review if their report is going to be published, but we believe that educating reviewers about the benefits of this model will increase participation. We’ll make sure we provide them with all the information they need to make an informed decision.

We’re all human and there’s still a lot of work to be done. But we know from research that diversity leads to better science. And, as a society publisher, it is our role to advance physics for the benefit of all.

Transparency versus anonymity in peer review is a false dichotomy. We believe that offering both double-anonymous peer review and full transparency post-publication allows for maximum objectivity, maximum accountability and minimum bias. And teaching authors and reviewers the benefits of both these new methods will help us raise consciousness of what affects our judgements and spread best peer review practice around the sector.

If we want to strengthen the foundations of science, we must ensure that the highest and fairest possible standards of peer review are applied to all published work.

Kim Eggleton is research integrity and inclusion manager at IOP Publishing.