Evidence and argument

A very interesting report of research on the link between guns and crime. The findings themselves are interesting but I find the approach even more interesting. As in many fields and around many controversies ( the link between taxes and growth, between regulation and innovation, between freedom and security, etc.) there is much argument supported by many reports of widely divergent quality. Everyone always has some reason for believing what they do and they will enthusiastically reach for the thinnest of straws. Given motivated reasoning, publication bias, and confirmation bias, there is a lot of cognitive pollution out there and sorting the wheat from the chaff, the sinners from the virtuous, the enlightening versus the deceptive is in itself an arduous task.

'Strongest' Research Shows No Link Between Gun Ownership Rates and Higher Crime by Ross Pomeroy covers such an effort by Gary Kleck on the vexed issue as to what might be the relationship between gun ownership and crime. Does a well armed society make a more civil society or does easy access to guns make society more dangerous? Valid questions with a lot of potential nuance and uncomfortable trade-offs.

I liked this articulation of the challenge.

"All research is flawed, and all bodies of research are incomplete," Kleck noted, "but that does not mean we cannot distinguish the less flawed work from the more flawed, and draw tentative conclusions based on the best available research conducted so far."

You might quibble with any number of aspects of his approach but it is a structured, pre-defined approach that has the virtue of being objective.

Kleck included 41 studies that examined the association between measured gun levels and crime rate in his analysis, then used three specific criteria to gauge the strength of the studies.

First, he looked for a validated measure of gun ownership. In-depth surveys and percent of suicides with guns were two of the few acceptable measures. Second, he checked to see if confounding variables were properly controlled for and how many were included. Third, he checked to see whether the researchers used procedures that would rule out reverse causality, i.e. whether crime rates actually caused gun ownership to increase. (Past studies have shown that when crime rises in an area, gun ownership often increases, likely for purposes of self-defense.)

Kleck's findings.

In all, the 41 studies produced 90 findings on gun ownership and various crime rates. Of these, 64% found no statistically significant positive affect between gun ownership and crime. However, 52% did identify a link between gun ownership and homicide.

When Kleck applied his three methodological criteria (valid measure of gun ownership, causality procedures, controlled for >5 confounding variables) to the studies, he found that the more criteria they met, the more likely they were to show no link between gun ownership and crime. The reversal was particularly noticeable for homicide. While 65% of the studies that met none of the criteria found a link between gun ownership and homicide, the three studies that met all of the criteria did not.

"The overall pattern is very clear – the more methodologically adequate research is, the less likely it is to support the more guns-more crime hypothesis," Kleck remarked.

This looks like very good work and probably moves the dial a bit on the evidentiary side of the argument. There is good reason to believe that it won't resolve the argument.

Arguments fall into a handful of types some of which can be resolved with clearer definitions and better evidence. Many arguments, however, cannot be resolved because they are not really causal or factual arguments which can be resolved through better quality facts and more robust empirical observation. Many arguments are normative in nature or are trade-off arguments.

Guns, gun control, and the link between guns and violence usually fall into the normative and trade-off categories of arguments. It helps to have good data and robust evidence but it doesn't address the real argument.

On the normative side, the contrasting arguments are roughly 1) guns are an instrument of violence and you cannot effectively fight violence with violence; it is simply wrong to allow society to be awash with guns. In contrast, the other side of the normative argument is something along the lines of 2) Gun ownership is an enumerated right in the Constitution and more broadly, everyone has the right to self-defense, that this is an inalienable right.

On the trade-off side, the contrasting arguments are more nuanced. There are those that make the argument (from a trade-off perspective) that even though "when seconds count, the police are only minutes away" might be true, that the individual losses that occur from defencelessness are less than the aggregate losses because of the greater prevalence of guns. The trade-off argument is that individuals have to give up the capacity to defend themselves because of a greater societal good. In contrast, there are those who are making a different trade-off argument. They are not focused on the tactical issue of crime control and dangers. They are focused on a more strategic issue, the capacity of the individual citizen to protect themselve from the government gone wrong. Their argument is not only that a well armed society is a more civil society (as in politeness between individuals) but also that a well armed citizenry makes for a, literally, more civil society. After all the checks and balances of federalism and separation of powers (legislative, executive, and judicial), the final backstop for protection of the individual and minority from the mob and the majority is that they might be well armed. It is a very American argument but with some good evidence to support it but ultimately it rests on a trade-off between current tactical goals (safety of the citizen from local dangers) and future strategic goals (safety of the citizen from systemic dangers such as rogue government, government overreaching, mob mentalities, etc.).

Evidence about the relative consequence of individual gun ownership has virtually no impact on gun arguments that are normative in nature and only marginal impact on arguments that pivot on trade-offs.