Usefully true probabilities

From Remembering a Crime That You Didn’t Commit by Douglas Starr.

In 1906, Hugo Münsterberg, the chair of the psychology laboratory at Harvard University and the president of the American Psychological Association, wrote in the Times Magazine about a case of false confession. A woman had been found dead in Chicago, garroted with a copper wire and left in a barnyard, and the simpleminded farmer’s son who had discovered her body stood accused. The young man had an alibi, but after questioning by police he admitted to the murder. He did not simply confess, Münsterberg wrote; “he was quite willing to repeat his confession again and again. Each time it became richer in detail.” The young man’s account, he continued, was “absurd and contradictory,” a clear instance of “the involuntary elaboration of a suggestion” from his interrogators. Münsterberg cited the Salem witch trials, in which similarly vulnerable people were coerced into self-incrimination. He shared his opinion in a letter to a Chicago nerve specialist, which made the local press. A week later, the farmer’s son was hanged.

Münsterberg was ahead of his time. It would be decades before the legal and psychological communities began to understand how powerfully suggestion can shape memory and, in turn, the course of justice.

Starr then provides some interesting evidence from various psychology studies that buttress the hypothesis that people are very sensitive to suggestion and, under the right circumstances, prone to making false confessions. He concludes,

Shaw and Porter’s study also provides further evidence of the inaccuracy and malleability of human memory, evidence that is already compelling enough to have persuaded the state supreme courts of New Jersey and Massachusetts to mandate that judges instruct juries that eyewitness testimony is inherently unreliable. “Evolutionary theorists say memory is good enough—just good enough for us to survive and to reproduce,” Shaw told me. “But, at the very least, this research calls into question whether we should be putting so much weight on any memory in court”—especially in the absence of corroborating proof. “It’s sort of a reality check.”

I don't disagree, human observation is subject to a wide range of material biases and flaws and human memory is measurably fallible.

Granted. But then what? Oh, we'll rely on physical evidence! But the collection, analysis and interpretation of physical evidence is as weak as human observation and human memory. The most recent systemic exposure of the questionable basis for interpreting fires. In the same magazine, The New Yorker, in 2009, there was an account, Trial by Fire, of an inmate executed in Texas for having killed his wife and children by arson. The article raises nightmarishly reasonable questions about the quality of the forensic evidence used to convict Todd Willingham, creating credible grounds for believing that Texas executed an innocent man falsely convicted of a horrific crime.

So we are left with good reason to believe that both eyewitness testimony and forensic evidence are flawed. If we can't believe eyewitness accounts or the testimony of forensic experts, how can we reasonably settle on guilt or innocence?

I believe part of the issue is that there is a whiff of advocacy journalism in both these accounts which overstate the problem. Yes, eyewitness accounts are known to be subject to a range of observational biases and memory failures. But not all eyewitness accounts are equally fallible. It is not a case of falsus in uno, falsus in omnibus. It is a case of probabilities. How probable is it that this eyewitness testimony is accurate? For example, in the Trayvon Martin case, all the ear-witnesses testified that they heard one shot fired. However, one ear-witness testified that she recalled hearing three shots fired. Hypothetically, it was possible that multiple witnesses were wrong and one was right. However, when you combined the earwitness testimony with the forensic evidence of a single bullet wound and a single shell casing from the gun, the probability of the statement "a single shot was fired" being true increases quite dramatically.

Ultimately, what both authors are doing is pointing out that evidence, whether eye-witness or forensic, is fallible and I believe that to be a good warning. The subtle, and perhaps not intentional, implication that because eyewitness and forensic evidence is fallible, therefore we should not use either, is throwing the baby out with the bath water.

The two articles, paired together, do serve as an interesting buttress to the proposition that all Truth is provisional on new evidence. To determine the truth of a proposition, we look at the internal logic of the argument, does the predicate logically lead to the conclusion? We then look at the empirical evidence of a proposition, are the known facts consistent with the argument; are there other facts that are inconsistent with it? Finally, we look at the rationality of the argument, does the internal logic and the external facts work together consistently to make it probable that the argument is true? Logic, Empiricism, Rationality - they are powerful tools for vetting an argument and arriving at a probabilistic estimation of the truth of the argument.

But it will always be a probability. In the Martin scenario, one wound, one shell casing, multiple ear-witnesses makes it highly probable that a single shot was fired. But it is not impossible to construct a scenario in which three shots were fired, two missed, two shell casings were disposed of and ear-witnesses were too influenced by one another to recall accurately. It is possible but improbable. We want certainty but all we have are probabilities. But usefully true probabilities.