Amgen launches new platform to help fix scientific ‘reproducibility crisis’

A major pharmaceutical company is launching a new platform for the publication of experiments that attempt to replicate previous research in a bid to tackle the “reproducibility crisis” in science.

In 2012, Amgen alarmed the scientific world by revealing that it had been able to reproduce the results of only six out of 53 “landmark” cancer studies. This confirmed similar, worrying findings from German drug company Bayer released the previous year.

Launched on 4 February, the Preclinical Reproducibility and Robustness channel is on the F1000Research publishing platform, which is open access and invites peer review after publication.

Bruce Alberts, a prominent biochemist at the University of California, San Francisco and former editor-in-chief of Science, said it would be a home for replication attempts that were often shunned by journals.

“One of the problems with science is that there’s a bias against publishing [studies] to reproduce” results, said Dr Alberts, who is helping to promote the channel.

So far, in-house attempts to reproduce results by drug companies had not generally been made public, he explained. The first papers published on the channel are three replication studies by California-based Amgen – which all fail to confirm the results of previous experiments.

The inability to reproduce results has been blamed on a number of factors, from statistically underpowered studies to the post-hoc manipulation of data.

Some have criticised high-profile journals like Science for exacerbating the reproducibility problem by claiming they demand novel, positive and media-friendly results, which can create pressures to come up with a certain experimental outcome.

Asked about the role of his former magazine, Dr Alberts said that “all journals only want positive results”.

“What you could say about these high-profile journals is that they want a finding that has a big impact”, something that could encourage dishonest scientists to hunt for a particular result, he said.

To deal with this, journals like Science needed a “very, very high standard for [peer] review”.

During his period at Science, Dr Alberts said he had resisted telling scientists that they would be published only if they completed further experiments and found a similar outcome, a practice he said was “very distorting” to the scientific process (since it could create an incentive to get the same result by whatever means).

But the pressures journals put on researchers was only a “small part of the problem”, he said. More important was setting better rules for proper scientific procedure.  

However, science should not aim for a 99 per cent reproducibility rate, he added. “You don’t want every paper to be perfect because that would delay publication” and stifle scientists’ ability to put out new ideas, he argued.

Dr Alberts said that unless the problem was addressed, there was “of course” a risk that scientists could lose public trust in the same way journalists and politicians had done.  

Like free market capitalism, “with no rules and oversight [science] becomes very distorted” and up until now there had not been enough efforts to fix problems. 

Marcia McNutt, editor-in-chief of the Science family of journals, said its track record showed it had been "a leader in raising standards for reproducibility and publishing reproducibility studies, including those with negative findings". She gave the example of a recent paper it published probing the replicability of 100 landmark papers in psychology.

"We absolutely want everything we publish to be interesting, groundbreaking, and thought-provoking, but we never demand positive results. Disproving a reigning paradigm might be just as interesting as breaking new ground,” she added.

david.matthews@tesglobal.com