Monday, October 26, 2020

They are bad science practice but they are not only bad science practice.

From Science Fictions: How Fraud, Bias, Negligence, and Hype Undermine the Search for Truth by Stuart Ritchie.  Page 26. 

Kahneman also reviewed research on ‘money priming’. In another Science paper, also from 2006, social psychologists had found that subtly reminding people about money – for instance, having them sit at a desk where there happened to be a computer showing a screensaver of floating banknotes – made them feel and behave as if they were more self-sufficient, and made them care less about others.  Being exposed to money priming, the authors said, made participants prefer ‘to play alone, work alone, and put more physical distance between themselves and a new acquaintance’.  Indeed, when asked to arrange the room’s seating for a face-to-face conversation with a stranger, the money-primed participants, compared to those who’d seen a blank screen, set their chairs almost 30 centimetres further apart. Quite an impact, you might think, for a simple screensaver. This was a pattern in the most prominent priming studies: very subtle primes appeared to cause impressive changes in the way people behaved.
 
Kahneman concluded that these kinds of priming studies ‘threaten our self-image as conscious and autonomous authors of our judgments and our choices’. He had little doubt about their soundness. ‘Disbelief is not an option,’ he wrote.  “The results are not made up, nor are they statistical flukes. You have no choice but to accept that the major conclusions of these studies are true. More important, you must accept that they are true about you.’
 
But perhaps Kahneman shouldn’t have placed such complete trust in these priming effects, despite their being published in one of the most renowned scientific journals. As it turns out, along with the discovery of Diederik Stapel’s fraud and the publication of Daryl Bem’s weird psychic results, it was a priming study – or, rather, the attempted replication of one – that was among the initial spurs of what has become known as ‘the replication crisis’. 
 
In that original priming study, the researchers asked participants to find the odd one out from a jumbled list of words, the rest of which could be rearranged to form a sentence. For half of the participants, the odd-words-out were random and neutral; for the other half, these words had to do with elderly people. These included, for example, old, grey, wise, knits, and Florida – the last of which is “known in the US for having a large population of retirees. Having completed the task, the participants were free to leave – but unbeknownst to them, the experimenters were timing how quickly they walked down the corridor to leave the building. Demonstrating again the mental connection between concepts and actions, the participants who had been primed with elderly-related words walked more slowly out of the lab than those in the control group.
 
Published in 1996, this study has now been cited over 5,000 times by other researchers and has become a staple of psychology textbooks – I remember being taught about it myself when I was a student. In 2012, though, an independent group tried running exactly the same experiment again, but with a bigger sample size and better technology. They found no differences in walking speed. They proposed that the original study might have come up with its results because the research assistants, who timed the participants with stopwatches, knew which participants were expected to behave in which way, possibly influencing their timing. Measuring the participants’ walking speed with infrared beams, as was done in the replication study, appeared to nullify the supposed priming.  Within a few “years, other labs tried to replicate both the Macbeth effect and the money-priming effect, also in much larger, more representative samples.  These efforts also conspicuously failed. There’s no reason to think, to use Kahneman’s terms, that the various priming results were ‘made up’; we have to assume they were arrived at in good faith. But ‘statistical flukes’? Perhaps exactly that.

Again, I remember these experiments and how seriously they were credited when even a mite of good sense, statistics and skepticism suggested otherwise.  And when you have the imprimatur of a Kahneman, a leader in his field, proclaiming that disbelief was not an option, well that is a pretty powerful injunction to stay in your lane and be quiet about the fact that the emperor had no clothes.  These studies were bunk from start to finish and yet they were lauded and peddled by those who should have known better.

One thing which is missing from Ritchie's book is the political/ideological context.  These priming experiments were being conducted in the years when Critical Theory, Critical Race Theory and Postmodernism were also rising.  By law and practice, overt racism has been declining in the US for decades, as has been racial hate crimes.  Most income and academic gaps are easily identifiably due to differences in personal choices and basic factors such as family structure, IQ, and the like.  

Critical Theory, Critical Race Theory and Postmodernism are pervasive in academia, particularly in the field of sociology and psychology.  With declining empirical evidence of racism, academics were desperate to find ways to demonstrate systemic racism.  Evidence which could be plausible but not obvious.  Priming studies fit the bill nicely. 

Ritchie treats them solely as products of bad science practice, and they are that.  But they are not only that.  They are bad science practice in service of propping up an otherwise demonstrably false ideological belief.  


No comments:

Post a Comment