We can get on with the work of improving our epistemic environment.

An additional example of increasingly questionable academic quality.  From Age of Invention: Does History have a Replication Crisis? by Anton Howes.  He begins by referencing the 2011 replication crisis in the hard sciences and then transitions to his field, history.

But I’ve become increasingly worried that science’s replication crises might pale in comparison to what happens all the time in history, which is not just a replication crisis but a reproducibility crisis. 

[snip]

Historical myths, often based on mere misunderstanding, but occasionally on bias or fraud, spread like wildfire. People just love to share unusual and interesting facts, and history is replete with things that are both unusual and true. So much that is surprising or shocking has happened, that it can take only years or decades of familiarity with a particular niche of history in order to smell a rat. Not only do myths spread rapidly, but they survive — far longer, I suspect, than in scientific fields.

Take the oft-repeated idea that more troops were sent to quash the Luddites in 1812 than to fight Napoleon in the Peninsular War in 1808. Utter nonsense, as I set out in 2017, though it has been cited again and again and again as fact ever since Eric Hobsbawm first misled everyone back in 1964. Before me, only a handful of niche military history experts seem to have noticed and were largely ignored. Despite being busted, it continues to spread. Terry Deary (of Horrible Histories fame), to give just one of many recent examples, repeated the myth in a 2020 book. Historical myths are especially zombie-like. Even when disproven, they just. won’t. die.

Or take the case of the 12,000-franc prize instituted by Napoleon for an improved method of preserving food for the use of his armies, which prompted Nicolas Appert to invent canned food. It’s frequently cited to show the how prizes can have a significant impact. Except that, despite being repeated hundreds of times, it literally never happened. Appert was given money by the French government, but it was a mere reward in recognition of his achievement, given over a decade after he had invented the method. The myth of the food canning innovation prize is a truly ancient one, which I traced back to a mis-translation of a vaguely-worded French source all the way back in 1869. That’s over 150 years of repeated falsehood, and I see no signs of it slowing.

More than four decades ago, I had a revelatory experience in a class in my junior or senior year.  Regrettably I recall neither the name of the class or the name of the teaching assistant.  It was an economic development class perhaps or possibly government.  

The TA had given us an assignment to read and critique an assigned paper that was only 10-20 pages long in which the author made an argument.  We were to come to class prepared to discuss our critique. 

We all came in prepared and had a reasonably robust discussion over the first fifteen minutes or so of the class.  Interesting, but nothing out of the ordinary.

But then the TA did something very out of the ordinary.  We had all read the 20 pages and taken the sourcing at face value.  There was perhaps a page or two worth of footnotes at the end, providing source referencing to support the argument in the paper.  We had read the paper and never looked at the sources.

The TA closed the conversation about the argument and turned our attention to the footnotes.  Patiently, one by one, he showed us the actual source in the footnote versus how it was referenced in the text.  Time and again, the footnote was irrelevant, erroneous, or misinterpreted.  We had accepted that the building blocks of the argument were all correct and had been critiquing the assemblage of those blocks into an argument.

The TA's point, masterfully made, was that most of the building blocks were not actually building blocks at all.  That once we stripped out all the elements of the argument which had no source or were irrelevant, misinterpreted or erroneous, there was really hardly an argument at all.  We had wasted our intellectual energy by attacking a perceived narrative rather than doing the hard work of checking the facts first.

Howes proceeds to offer an example in his column.  

This lack of effective institutions or incentives was really brought home to me recently by the publication of a paper in the prestigious journal History & Technology by Jenny Bulstrode of UCL, in which she claimed that the inventor Henry Cort had stolen his famous 1783 iron-rolling process from Reeder’s iron mill in Jamaica, where it had been developed by 76 black metallurgists by passing iron through grooved sugar rollers. It was a widely-publicised paper, receiving 22,756 views — eleven times as many views as the journal’s next most most read paper, and frankly unheard of for most academic papers — along with a huge amount of press coverage.

Bulstrode argued that Cort had heard of the invention via a relative, the master of the ship Abby, who had been in Jamaica and in November 1781 visited him in Portsmouth; that in March-May 1782 Reeder’s mill was destroyed by the British army on the pretext that it might be used for weapons in a slave revolt during wartime, but that this was really at the behest of Cort to destroy the competition; and that the grooved rolling machines at Reeder’s mill were dismantled and sent to Portsmouth where Cort could use them.

Howes demonstrates that this argument by Bulstrode was a complete muddle of cognitive pollution.  Incorrect timelines, factual errors, leaps of imagination, etc.  What looked like a properly researched history paper was, when you did the hard work of going back to the source information, just a load of tosh or ideological tripe.  

It is easy to feel like we are at a precipice, beginning a long and perhaps accelerating slide into decline.  It feels as if cognitive pollution is everywhere and getting worse.  Activists and ardent ideologues advance self-evident nonsense and expect everybody to be bullied into accepting it.

I suspect that we are actually in a different metaphor.  We have been slogging through a moorland of wetlands and ridges, mixed ecologies, with little idea of where we are heading and no path to get there.

We are given a compass and a topographical map that shows us the sloughs to avoid, the hills that don't necessarily need to be climbed natural pathways and, perhaps most importantly, the larger view of the entire landscape that allows us to know where we need to be headed.  

For fifty or seventy-five years, we have subsidized massive knowledge creation through universities but with exceedingly weak quality control.  Hundreds of billions of dollars for creating research but virtually nothing to ensure that any of that research which was produced was actually usefully true.  

Arnold Kling frequently observes that American government policy in the market place is to effectively restrict supply and subsidize demand, leading to inflation, shortages, and corruption.  In contrast, in academia, a non-market economy, we seem to have the reverse government policy.  Policies which subsidize supply while restricting demand (most papers are cited fewer than five times.)  

We are in a bad place from an historical perspective.  We have made the moorlands and wetlands and marshes of knowledge much more difficult to traverse through bad policy.  

On the sunny side, though, what we are now seeing is that we are indeed in bad terrain but because of a  more engaged public and better knowledge access (internet, smartphones, et al), the terrain is getting clearer.

We have a lot of clean-up to do, some wasteland rehabilitation, some some brown site restoration, some drainage of marshes.  But the internet and universal knowledge access serve as a compass and map.  We will still occasionally wander into muddy waters and barren fells but now we are in a better position to spot them and size them and avoid them.

There is a lot of cognitive pollution and we can bewail just how much there is.  Or we can celebrate that we are now beginning to realize just how weak was our understanding of how extensive was the cognitive pollution.

Now that we know, we can be more rigorous, more skeptical, more diligent.  

Institutions will continue to subsidize the production of low quality "knowledge," incentives will still reward cognitive pollution over clean-up work, the mainstream media will still celebrate the improbable and ideological over the useful.  

But we can see that as our challenge better now than we did ten years ago.  Just because we expected better from academia doesn't mean we deserved better.  It is what it is and we can see that more clearly.  And we can get on with the work of improving our epistemic environment.  Rather than simply and blindly remaining ignorant of the torrent of cognitive pollution spewing forth.