Friday, July 2, 2021

The very traits that put these experts in the public eye made them the worst at prediction

From Enlightenment Now by Steven Pinker.  

Indeed, the very traits that put these experts in the public eye made them the worst at prediction. The more famous they were, and the closer the event was to their area of expertise, the less accurate their predictions turned out to be. But the chimplike success of brand-name ideologues does not mean that “experts” are worthless and we should distrust elites.  It’s that we need to revise our concept of an expert. Tetlock’s superforecasters were:

pragmatic experts who drew on many analytical tools, with the choice of tool hinging on the particular problem they faced. These experts gathered as much information from as many sources as they could. When thinking, they often shifted mental gears, sprinkling their speech with transition markers such as “however,” “but,” “although,” and “on the other hand.”  They talked about possibilities and probabilities, not certainties. And while no one likes to say “I was wrong,” these experts more readily admitted it and changed their minds.

Successful prediction is the revenge of the nerds. Superforecasters are intelligent but not necessarily brilliant, falling just in the top fifth of the population. They are highly numerate, not in the sense of being math whizzes but in the sense of comfortably thinking in guesstimates. They have personality traits that psychologists call “openness to experience” (intellectual curiosity and a taste for variety), “need for cognition” (pleasure taken in intellectual activity), and “integrative complexity” (appreciating uncertainty and seeing multiple sides). They are anti-impulsive, distrusting their first gut feeling. They are neither left-wing nor right-wing. They aren’t necessarily humble about their abilities, but they are humble about particular beliefs, treating them as “hypotheses to be tested, not treasures to be guarded.” They constantly ask themselves, “Are there holes in this reasoning? Should I be looking for something else to fill this in? Would I be convinced by this if I were somebody else?” They are aware of cognitive blind spots like the Availability and confirmation biases, and they discipline themselves to avoid them. They display what the psychologist Jonathan Baron calls “active open-mindedness,” with opinions such as these:

 People should take into consideration evidence that goes against their beliefs. [Agree]

 It is more useful to pay attention to those who disagree with you than to pay attention to those who agree. [Agree]

 Changing your mind is a sign of weakness. [Disagree]”

“Intuition is the best guide in making decisions. [Disagree]

 It is important to persevere in your beliefs even when evidence is brought to bear against them. [Disagree]

Even more important than their temperament is their manner of reasoning. Superforecasters are Bayesian, tacitly using the rule from the eponymous Reverend Bayes on how to update one’s degree of credence in a proposition in light of new evidence.

From this we might conclude that there are no superforecasters at CDC, WHO, FDA or in the corridors of power then.   


No comments:

Post a Comment