Psychologists reluctant to own up to research mistakes

Most academics agree that science would benefit if researchers owned up to their mistakes a little more readily. But, when you ask scholars to publicly confess their errors, it turns out that most would still rather keep shtum.

That is according to psychologists who led a major drive to persuade their colleagues to state which of their published findings they no longer stood by – and could only persuade the authors of six papers to take part.

Psychology has been gripped by the same concerns over reproducibility that have engulfed other disciplines, with one 2015 study finding that the results of fewer than half of 100 prominent psychology papers could be replicated, although the methodology of this experiment itself was in turn challenged.

The Loss-of-Confidence Project, led by academics at eight institutions in Europe, North America and Asia, asked psychologists around the world to name articles that they now recognised as having been undermined by theoretical or methodological problems with the study design or data analysis.

The project’s founders argue that authors are best placed to make corrections, avoiding the need for costly replication studies and for the confrontations that occur when academics challenge the rigour of fellow scholars’ work.

Authors who responded submitted statements declaring that they had lost confidence in their conclusions because of model misspecifications, invalid inferences or inadvertent p-hacking, having presented selected data as being statistically significant when in fact there was no real underlying effect.

But Julia Rohrer, a PhD student at the International Max Planck Research School on the Life Course in Berlin, who managed the project, said that the low number of responses suggested that there was a gap between psychologists’ ideals and their actual behaviour.

Some researchers might feel that issuing self-corrections would damage their careers or lead to questions over their other publications, but the experience of academics who have made such admissions in the past shows that it usually receives a positive response and “sends a strong signal you care about the accuracy of your work”, Ms Rohrer told Times Higher Education.

“On the one hand, we are very aware that errors and mistakes are such a common thing in science, but at the same time it is not being talked about and people act as if what they said a few years ago still stands – it’s very much a norm that you defend your findings no matter what,” she said.

“I think this discrepancy harms science and leads to a lot of wasted resources. We can make science more efficient and probably more enjoyable for the people involved if the personal stakes [of corrections] were not so high.”

The project’s founders argue that publishers could help academics by introducing a new label for retractions initiated by the original authors – for example, “voluntary withdrawal” – and that scientific societies could consider whether they wanted to acknowledge efforts by scholars to correct their own work more formally.

The project, which is outlined in a preprint on the PsyArXiv server, is still seeking self-corrections via www.lossofconfidence.com.

chris.havergal@timeshighereducation.com