Science’s decision to correct – but not retract – a high-profile paper on a potential HIV treatment has raised important questions about what journals should do when replication studies fail, according to experts.
In an editorial published on 6 September, Science editor Jeremy Berg explained that the magazine was issuing an official correction to a 2016 study into SIV – an HIV-like virus that can affect monkeys and apes – which indicated that combining an antibody with antiretroviral treatment could keep virus levels low in non-human primates.
The result led to a clinical trial on HIV patients of a similar drug, as well as three replication studies – all of which failed to reproduce the results of the initial study, which was led by Siddappa Byrareddy, an associate professor at the University of Nebraska Medical Center.
In March 2019, Science issued an editorial expression of concern after it emerged that one of the co-authors had used a slightly different strain of the virus than that reported in the study, but did not communicate this to his colleagues. The researcher had believed that strain was a better model for chronic HIV infection, Science said.
In his editorial, Professor Berg said that this “lack of clarity is an unacceptable communication failure that affected interpretation of the results both before and after publication”.
But the issue of the three failed replication attempts illuminated “challenges about replication” – a process he called the “cornerstone of science”.
“These complicated biological studies involve many variables in addition to the viral strain, including the immunological experiences of the experimental animals, the methods used for viral exposure, and other factors,” he said. “Even matching these factors as closely as possible, hidden variables might exist that cause differences.”
Dr Byrareddy told Times Higher Education that each replication study differed in significant ways to his original study. One conducted by Harvard University scientists “utilised a completely different virus” that was introduced intro-rectally, rather than intravenously, while the doses used of the virus were “extremely high”, he said.
“While the results of the three studies are important, the scientific community needs to look at these data carefully,” he said.
Using chronically infected HIV patients in the clinical trial was also “inappropriate” as previous tests had already shown that the treatment had no effect on monkeys chronically affected by the SIV virus, he added.
David Sanders, an associate professor in Purdue University’s department of biological sciences, said the error that led to Science’s correction may have related to the “well-known phenomenon” of “variation between viral isolates”.
“Something designated SIVmac239 or SIVmac251 may have a slightly different sequence in one lab compared to something with the same designation in another lab,” he said. “Obviously the authors should all have been made aware that this was the isolate that was being employed in the experiments.”
While a “thorough investigation” was required, however, “it is not normally grounds for retraction if the results of complex experiments cannot be reproduced if there is no evidence of malfeasance”, Dr Sanders added.
“The fact that there was both misreporting and irreproducible results does not necessarily mean that there is an infraction of scientific integrity that warrants retraction,” he said.
