Can we trust scientists?

Science Fictions

Exposing Fraud, Bias, Negligence and Hype in Science

Stuart Ritchie

Bodley Head 2020

Hb, 353pp, £18.99, ISBN 9781847925657

Something is rotten in the state of science. Some papers report results that are “seemingly impossible” and occasionally “outright fraudulent”. Books and media reports of robust, reputable studies can still be “incorrect, exaggerated, or dramatically misleading”. In “the very worst cases”, the fraud, bias, negligence and hype that’s all too common in science means that, this important and, because of Covid-19, timely new book warns, “people are dying”.

Science projects an aura of robust, reliable rationality. Reproducibility, for example, ensures the finding is real and not due to chance or human or technical error. But researchers “subjectively rated” that they replicated the results from only 39 per cent of 100 studies published in three leading psychology journals. On average, the replicated effect was only half that in the original study. Moreover, 97 per cent of the original studies had statistically significant results: there was a less than a one in 20 chance the results were a fluke. (Stuart Ritchie offers an excellent explanation of the use and abuse of statistical significance.) This declined to 36 per cent in the replicated studies.

As Ritchie notes in this insightful, engaging and readable book, the replication rate in other social science and psychological studies ranges from 38 to 62 per cent. (Fittingly, Ritchie is a psychologist. In many ways, psychology is in the forefront of attempts to address science’s issues.) A couple of drug companies found that they could replicate just 11 and 20 per cent of their preclinical studies (experiments, such as those on animals, before a medicine reaches humans). To allow replication, a paper is supposed to give full details of the method. But only one of 268 biomedical studies, including clinical trials, reported their full protocol. We can’t, of course, replicate ‘Oumuamua’s transit or Schliemann’s excavation of Troy. Nevertheless, a full protocol helps you decide how “robust” the study really is.

Richie uses a telling montage of anecdotes, aptly focusing