Time-travelling porn

From Science Fictions: How Fraud, Bias, Negligence, and Hype Undermine the Search for Truth by Stuart Ritchie.  Page 1.

“It is the peculiar and perpetual error of the human understanding to be more moved and excited by affirmatives than by negatives.

Francis Bacon, Novum Organum (1620)

January 31, 2011 was the day the world found out that undergraduate students have psychic powers.

A new scientific paper had hit the headlines: a set of laboratory experiments on over 1,000 people had found evidence for psychic precognition – the ability to see into the future using extrasensory perception. This wasn’t the work of some unknown crackpot: the paper was written by a top psychology professor, Daryl Bem, from the Ivy League’s Cornell University. And it didn’t appear in an obscure outlet – it was published in one of the most highly regarded, mainstream, peer-reviewed psychology journals.  Science seemed to have given its official approval to a phenomenon that hitherto had been considered completely impossible. 
 

At the time, I was a PhD student, studying psychology at the University of Edinburgh. I dutifully read Bem’s paper. Here’s how one of the experiments worked. Undergraduate students looked at a computer screen, where two images of curtains would appear. They were told that there was another picture behind one of the curtains, and that “they had to click whichever they thought it was. Since they had no other information, they could only guess. After they’d chosen, the curtain disappeared and they saw whether they’d been correct. This was repeated thirty-six times, then the experiment was over. The results were quietly stunning. When a picture of some neutral, boring object like a chair was behind one of the curtains, the outcome was almost perfectly random: the students chose correctly 49.8 per cent of the time, essentially fifty-fifty. However – and here’s where it gets strange – when one of the pictures was pornographic, the students tended to choose it slightly more often than chance: 53.1 per cent of the time, to be exact. This met the threshold for ‘statistical significance’. In his paper, Bem suggested that some unconscious, evolved, psychic sexual desire had ever-so-slightly nudged the students towards the erotic picture even before it had appeared on screen.
 
Some of Bem’s other experiments were less explicit, but no less puzzling. In one of them, a list of forty unrelated words appeared on the screen, one at a time. Afterwards came a surprise memory test, where the students had to type in as many of the words as they could remember. At that point, the computer randomly selected twenty of the words and showed them to the students again. Then the experiment ended. Bem reported that, during the memory test, the students were more likely to remember the twenty words they were about to see again, even though they couldn’t have known – except by psychic intuition – which ones they were going to be shown. This would be a bit like studying for an exam, sitting the exam, then studying again afterwards, and that post-exam study somehow winding its way back in time to improve your grade. Unless the laws of physics had suddenly been repealed, time is supposed to run in only one direction; causes are supposed to come before, not after, their effects. But with the publication of Bem’s paper, these bizarre results were now a part of scientific literature.
 
Crucially, Bem’s experiments were extremely simple, requiring nothing more complicated than a desktop computer. If Bem was right, any researcher could produce evidence for the paranormal just by following his experimental instructions – even a PhD student with next to no resources. That is what I was, so that is exactly what I did. I got in touch with two other psychologists who were also sceptical of the results, Richard Wiseman of the University of Hertfordshire and Chris French of Goldsmiths, University of London. We agreed to re-run Bem’s word-list experiment three times, once at each of our respective universities. After a few weeks of recruiting participants, waiting for them to complete the memory test and then dealing with their looks of bewilderment as we explained afterwards what we’d been looking for, we had the results. They showed … nothing. Our undergraduates weren’t psychic: there was no difference in their recall of the words presented after the test. Perhaps the laws of physics were safe after all.
 
We duly wrote up our results and sent the resulting paper off to the same scientific journal that had published Bem’s study, the Journal of Personality and Social Psychology. Almost immediately the door was slammed in our faces. The editor rejected the paper within a few days, explaining to us that they had a policy of never publishing studies that repeated a previous experiment, whether or not those studies found the same results as the original.
 
Were we wrong to feel aggrieved? The journal had published a paper that had made some extremely bold claims – claims that, if true, weren’t just interesting to psychologists, but would completely revolutionise science. The results had made their way into the public domain and received significant publicity in the popular media, including an appearance by Bem on the late-night talk show The Colbert Report where the host coined the memorable phrase ‘time-travelling porn’.  Yet the editors wouldn’t even consider publishing a replication study that called the findings into question.

Ritchie's book is full of these cases.   I remember when this study came out.  Already alert to cognitive pollution, I ignored the paper and watched it tumble over time.  Strong claims need strong evidence and in this instance overturning known physics required something more than an underpowered research with weak effect in a field noted for p-hacking and non-replication.  None-the-less for a couple of months, time-traveling porn was the rage.  Cognitive pollution indeed.