The mean time for clearing cognitive pollution is too long

A great example of good journalism, Mass Shootings in America: Anatomy of a Hyped Statistic by Carl M. Cannon. In the gun control debate, there are a plethora of studies, usually ill-designed and underpowered from both ends of the debate. Much of the debate hinges on careful parsing of definitions to hide inconvenient facts.

As an example, in the US, much of the focus for gun control advocates is the banning of long guns such as the AR-15, commonly described by journalists as an assault-rifle. Why this has become the focal point of their efforts is unclear as the AR-15, and indeed all long guns together such as hunting rifles and shotguns, account for fewer than 650 of some 15,000 murders in an average year. More people are murdered by kicking or being punched to death. After handguns, accounting for nearly 50% all murders, the next most common weapon of choice are knives, accounting for roughly 20% of all murders.

So why the careful parsing of weapons to distinguish AR-15s from long guns and long guns from all guns and shootings from all deaths by violence? Why the focus on AR-15s or long guns in general when they are such a negligible percent of the tragedy? There is no good explanation. Possibly ignorance. Possibly simply that long guns look more dangerous. Hard to tell.

My point is that definitions are the critical lens by which the numbers are viewed.

Cannon is addressing a couple of fairly shocking incidents of misleading research that have come out in the past couple of weeks. One incident was uncovered by NPR of all news outlets.

In April of this year, which is to say two months after the tragedy at Stoneman High School in Parkland, Fla., and a month before the Santa Fe High School rampage in Texas, the U.S. Department of Education published a booklet stating that during the 2015-2016 academic year “nearly 240 schools in this country…reported at least 1 incident involving a school-related shooting.”

Here was a truly shocking number. It was so high that National Public Radio attempted to contact each one of them. The results? NPR could confirm exactly 11 school shootings. Two-thirds of the schools they called reported definitively that they’d had no such incident. The federal government’s figure is bogus.

When that DOE report was issued in April, it received a lot of publicity and immediately was entered as a new fact in the debate about guns and school shootings. With dozens of mainstream media outlets, why did it take a third of a year before anyone questioned the "fact"? When 160 of the 240 schools affirmatively confirm that there were no such claimed incidents, it becomes hard to see how this was research rather than something more sinister. This is made up information, not misreported information.

The greater substance of Cannon's article is about an even more egregious news failure.

Three years ago, which means it was before the mass murders at the Pulse nightclub in Orlando and First Baptist Church of Sutherland Springs -- but after Sandy Hook Elementary and Fort Hood -- a University of Alabama professor sought to answer a chilling public policy question: Do countries other than the U.S. experience anything approaching America’s mayhem at the hands of shooters who randomly slaughter people in public places?

The researcher’s name is Adam Lankford, and his answer was unequivocal. No, it only happens here, he proclaimed in a much-quoted 2016 academic paper. America, he said, stands alone. From 1966 to 2012, he documented 90 mass shootings in the United States. In the rest of the world combined – Lankford said he canvassed 171 other nations – there were 202 mass shooters, meaning that the overwhelming number of nations had, on average, one mass killing.

It also meant that the United States, with less than 5 percent of the world’s population, had 31 percent of all the mass shootings.

“This is not a matter of opinion; this is a matter of applying statistical models to data from all these 171 countries,” Lankford said at the time. He explained that he compared national homicide rates, suicide rates, gross domestic product statistics, level of urbanization, and even the balance of men and women in each population. None of these factors proved relevant, he reported, which he said surprised him. Only one variable proved significant, he said: the availability of guns.

“The difference between us and other countries, [which] explains why we have more of these attackers, was the firearm ownership rate,” he said. “In other words: firearms per capita. We have almost double the firearm ownership rate of any other country.”

This conclusion, which is neither counterintuitive nor surprising, was referenced by President Obama. Lankford’s study was also showcased on the front pages of newspapers here and around the world, trumpeted by online news outlets and broadcast networks, invoked by pundits and politicians, and passed along on social media. It is revived, uncritically, every time there is a new mass shooting.

5% of the global population and 31% of global mass shootings is a compelling fact. Except that it isn't.

Lankford's claims were dismissed without refutation by Second Amendment advocates and trumpeted by gun controllers, it has been a part of the national conversation about gun control for two and a half years and shouted from all the bully pulpits. But, based on research seeking to replicate the findings, it is grossly untrue.

Unlike Lankford, Lott has shared his definitions, his methodology, his data sources, and his analysis. He identifies concerns about data integrity and completeness and how those issues might affect the conclusions. As a good researcher would.

What he finds is that Lankford lied about his research and restricted his data so that he could make a point that would both denigrate America and compel gun control. Well, that not what Lott says at all but that is a not illogical conclusion based on what Lott does find.

So Lott undertook a study of his own, which was obtained late last month by RCP and is now available online. Titled “How a Botched Study Fooled the World About the U.S. Share of Mass Public Shootings: U.S. Rate Is Lower Than Global Average,” it looks at the years 1998 to 2012 – the last 15 years of Lankford’s time frame.

In it, Lott has some eye-opening statistics. For starters, in the just last 15 years of the 47-year period covered in the NYPD and Lankford reports, Lott found 1,448 mass public shootings -- and 3,081 shooters -- outside the United States. This means he discovered 15 times as many mass killers as Lankford in less than one-third the time frame.

It also means that instead of having 31 percent of the world’s mass shootings, the United States has fewer than 3 percent. The key takeaway here is that, with 4.4 percent of the world’s population, the U.S. has less than its share of mass murderers, a finding that utterly undermines the prevailing narrative.

The US is 58 out of the 89 countries for which data is available, well down the rankings. And while the most dangerous countries for mass murders might fall into the category our president once referred to as "s**tholes", it is worth noting that Norway, Finland, and Switzerland all had higher rates of death from mass shootings than the US.

As I say, research in complex systems always hinges on definitions and methodologies. Lott will almost certainly have made some mistakes, overlooked some sources of data, misunderstood some nuanced definitions. It happens. My point is that Lankford's numbers were suspect from the beginning (owing to absence of methodology and source data) and yet his conclusions have been accepted as gospel in many quarters for the better part of three years. Journalists reported what they wanted to be true and never explored any of the red flags.

Violence is a complex system, heavily influenced by random chance, personal psychologies, mental health policies, criminal law enforcement, social conflict, etc. If we want to reduce societal violence, we need more information. We need more accurate conceptual models, we need better data, we need better research. We cannot make informed decisions on emotional whim given that the consequences could be worse than the status quo.

The rapid escalation of murders in the US after the late 1960s caught policy-makers and social scientists by surprise and there is no shared consensus as to its source. Likewise, the plunge in murders in the US after the mid-1990s also caught policy-makers and social scientists by surprise. Some of this decline is reasonably accorded to improved policing methods and training, but much of the decline also remains unexplained.

Its a complex system. We don't understand it.

But our epistemic basis for public debate is skewed by vocal/ideological advocates from one extreme of the spectrum to the other, amplified by a heavily skewed mainstream media cocooned in a limited and culturally homogenous bubble.

We will always make errors of fact and interpretation. We are human. But as rich as we are, as much as we have invested in research institutions, and as large as the mainstream media is, we ought to have a much healthier epistemic environment than we currently do. We have all the components and all the resources necessary, but we are not doing a good job at surfacing, questioning, and disseminating useful information and interpretations.

The fact that it took three years to refute a widely touted study which had no methodology or publicly available data or four months to refute a glaring claim such as the DOE statistic is a shameful indictment of academia and mainstream media.

The fact that illuminating reporting such as Cannon's surfaces from a recent internet news aggregator is a sign of hope that the promise of new distributed models of news making might overturn the curdled establishment mainstream media and improve our public discourse.