Only 12% of new science claims are substantiated.

From Investigation fails to replicate most cancer biology lab findings by Helen Albert.  The sub-heading is  - The reliability of early-stage cancer biology research is called into question by an investigation that concludes more than half of experimental results can’t be replicated by independent scientists.

The paralytic culture of erroneous, fallible and manufactured science results first documented in the psychology and sociology fields is now being revealed in the hard sciences now.  It was earlier possible to assume that the problem was that lack of replication was simply because psychology and sociology are so soft.  Now, biology is showing the same failings, suggesting that there must be something else going on.  Presumably culture and incentives.

An eight-year-long investigation into the reliability of preclinical cancer biology research has found that fewer than half of the results published in 23 highly cited papers could be successfully reproduced.

Tim Errington, director of research at the Center for Open Science in Virginia – which conducted the investigation – says the original plan was to reproduce 193 experiments from 53 papers. But, as explained in one of two studies the team publishes today, this was reduced to 50 experiments from 23 papers.

“Just trying to understand what was done and reported in the papers in order to do it again was really hard. We couldn’t get access to the information,” he says.

In total, the 50 experiments included 112 potentially replicable binary “success or failure” outcomes. However, as detailed in the second study published today, Errington and his colleagues could replicate the effects of only 51 of these – or 46 per cent.

Only a 46% replication rate?  Woof.   

But it is even worse than 46%.  Science is about making a claim and then providing the evidence to support that claim.  In this report, 193 claims were made.  Only 26% of claims provided detailed evidence.  And of the 50 claims with detailed evidence, only 46% replicated.  Thus, of 193 claims made (experiments), only 24 claims out of 193 claims made, presented replicable evidence.  In other words, of the 193 experiments, only 12% were supported in detail and then replicated.  

At 46% replication, one could colloquially claim that claims made by expert scientists in a hard science field are hit-and-miss.   At 12%, one has to conclude that most new scientific claims are bogus.  Either insufficiently described as to constitute evidence.  Or if adequately described, still likely to fail to replicate.