Tell me the effect size!

I don't know if it is sloppy journalism, basic innumeracy, or something else, but it drives me crazy. The example is Why Blind People Are Better at Math by Diana Kwon. What is my beef? Effect size. When you say something is real or important, tell me how real and how important. Too many, perhaps most or close to all, journalists elide the effect size completely.

Earlier this week in Not race, but choices of city living, violence, and governance, I pointed out that Pew Research was making the argument that

And, as it turns out, black and white Americans tend to have very different views on what they think is motivating these demonstrations.

They implied that there was a difference in kind of opinions between blacks and whites when in fact it was only a difference in degree. For example, a large majority of blacks and whites both believed that black protests were motivated by a desire to hold police accountable for their actions. Same belief, large majorities. However, 63% of whites believed this but even more, 79% of blacks believed it. At least in that article, they provided the numbers so that the reader could understand the real message and work out the actual effect size.

Kwon's article looks interesting. I didn't know that blind people were better at maths. And it is interesting to speculate why that might be. The article indicates that the number one candidate is that blind people have to improve their working memory capacity as compensation for the loss of their visual sense and that this compensation is likely a contributor to improved maths facility.

Is there something that allows the blind to excel? The leading theory is that because they cannot rely on visual cues or written materials to remember things, they develop stronger working memory than the sighted, which is critical to doing well at math. Another potential explanation is that because blind children spend a lot of time touching and manipulating objects, they learn to interpret numerical information with multiple senses, giving them an advantage.

BUT. Tell me more about the originating proposition before you get to the causes. Kwon simply assumes into existence that blind people are better at maths than the sighted. I am happy to entertain the notion, but show me that it is true first.

Kwon offers a handful of instances of gifted mathematicians who were also blind. Fine. That proves nothing other than that blindness is not a barrier to that maths ability. But her argument is much more than that. Kwon argues that blind people are better than the sighted at maths. Show me. There are two possible forms of this argument. The strong form is that all blind people are better at maths than any sighted person. That is easily refuted.

The weaker form of the proposition, and the more common, is that on average, blind people are better at maths than sighted people. OK, what is the evidence for that. And again, what is the effect size? Are they, on average, 1% better? 5%? 10%? 20%? The smaller the effect size, the less interesting. If there is only a 1% difference in some standardized maths score, then almost certainly there is no real issue. It is likely within the margin of error and the difference is attributable to sample size, or inadequate randomizing of the sample, or differences in testing procedures between sighted and blind. If the claim is that there is a 10 or 20% difference, then that is fascinating. If the effect is real and is sizable, then I want to know more about the causes.

However, as a journalist, don't skip over whether the effect is real or sizable and go straight to the possible causes. Show me the effect is real before telling me what might be causing it. Regrettably though, most journalists dispense with justification and skip, as Kwon does, straight to speculation. The whole premise of the article rests on (emphasis added to weasel words):

A number of studies suggest that perhaps both conditions are at play. In the early 2000s, Julie Castronovo, along with a group of psychologists at the Université Catholique de Louvain, in Belgium, conducted some of the first investigations to test the basic numerical abilities of the blind. To their surprise, they found that not only were these individuals unimpaired, the average blind subject possessed even sharper skills than the average test subject who could see.

How many investigations? How many subjects? How many study controls? What types of controls? What was the average difference in scores (the effect size)? Kwon is silent on all this. Without knowing that the effect is real and material, the article is worthless.

If it is real and material, there are all sorts of pedagogical and other implications. But we won't know from Kwon.