Simpson's Paradox

A couple of years ago I referred to The Texas-Wisconsin Paradox and intergenerational income mobility. The Texas-Wisconsin paradox became a point of some marginal discussion in the campaign led by teacher's unions to recall Wisconsin governor Scott Walker in 2013. One of the claims made by the Wisconsin teachers was that unions were responsible for higher student performance in Wisconsin when contrasted with non-unionized education in Texas.

In the internet age, this claim had a half-life of about 12 hours before it was pointed out that while the overall average for students in Wisconsin was higher than the overall average for students in Texas, when you split out the racial groups (White, Blacks, Hispanic, Asian-Americans, Native Americans), each racial group in Texas scored higher than its corresponding group in Wisconsin. Not knowing what to call this, I referred to it as the Texas-Wisconsin paradox where the score at the aggregate level can be higher but all the sub group score averages are lower.

In the case of Texas, all its racial groups scored higher than the corresponding groups in Wisconsin. However, the two highest scoring groups, Whites and Asians, were heavily overrepresented in Wisconsin and the lowest scoring groups, African-Americans and Hispanics, were overrepresented in Texas. Texas teachers were doing a good job of educating the students they were faced with, better than teachers in Wisconsin, but they were faced with a different mix of students.

I now discover that this is referred to as Simpson's Paradox, a discovery I made from When average isn't good enough: Simpson's paradox in education and earnings by Brad Hershbein. Hershbein provides an earlier example.

In the early 1970s, the University of California, Berkeley was sued for gender discrimination over admission to graduate school. Of the 8,442 male applicants for the fall of 1973, 44 percent were admitted, but only 35 percent of the 4,351 female applicants were accepted. At first blush, and assuming the applicants’ qualifications were similar, this pattern indeed appeared consistent with gender discrimination. However, when researchers looked more closely within specific departments, this bias against women went away, and even reversed in several cases.

This apparent contradiction, in which the trend of the whole can be different from or the opposite of the trend of the constituent parts, is often called Simpson’s paradox, after British statistician Edward H. Simpson, who described the phenomenon in 1951. In the Berkeley case, the “paradox” occurred because women disproportionately applied to departments with low acceptance rates, as shown in the table above, while men disproportionately applied to departments with high acceptance rates. Examples of Simpson’s paradox have also been found in baseball batting averages, on-time flights of airlines, and even survival rates from the Titanic.

Glad to make the discovery. I was sure that there had to be a proper name for the statistical phenomenon but simply wasn't googling the right questions in the right fashion to find the answer.