Every one of those common assumptions is simplistic, misguided, or downright wrong

A week or two ago I had a post, The Texas-Wisconsin Paradox talking about how comparisons between systems is very dependent on comparing apples-to-apples.

As if to drive home the point, there is a new study examining the PISA international education results. Despite spending more per student than anybody else, the US usually ends up somewhere in the middle on math and reading scores. In my paradox post I mentioned this in passing and indicated my skepticism that PISA was adequately taking into account the cultural heterogeneity of the US.

As if to validate my point, Derek Thompson summarizes a new study in his post, Why Gloomy Pundits and Politicians Are Wrong About America's Education System .

Here's what everybody knows about education in the United States. It's broken. It's failing our poorest students and codding the richest. Americans are falling desperately behind the rest of the developed world.

But here's what a new study from the Economic Policy Institute tells us about America's education system: Every one of those common assumptions is simplistic, misguided, or downright wrong.

When you break down student performance by social class, a more complicated, yet more hopeful, picture emerges, highlighted by two pieces of good news. First, our most disadvantaged students have improved their math scores faster than most comparable countries. Second, our most advantaged students are world-class readers.

Why break down international test scores by social class? In just about every country, poor students do worse than rich students. America's yawning income inequality means our international test sample has a higher share of low-income students, and their scores depress our national average. An apples-to-apples comparison of Americans students to their international peers requires us to control for social class and compare the performances of kids from similarly advantaged and disadvantaged homes.

It doesn't come out all that clearly in the post but apparently most other countries are oversampling participants in the tests from the higher income/social classes, groups who normally perform better on such tests. So, if I am reading the study correctly, the US had 20% of its test taking participants from the bottom quintile of income; what you would expect. On the other hand, Finland only had 6% of test takers from the bottom quintile. Of course their average is going to be higher.

This reminds me of the first time I encountered this apples-and-oranges issue as it relates to education measurement. I was in college and had only been back in the US three or four years and was still learning much information which my peers took for granted. I was particularly struck by the fact that Iowa had such an outstanding educational system based on a comparison of their SAT results to those achieved by other states. It was a couple of years before I discovered that in addition to SATs there was another standardized test, the ACT. In Iowa, as it turned out, everyone took the ACT but only those going on to college, and in particular those going out of state to university, would take the SAT. The consequence was that Iowa was reporting an SAT average for the top 5% of their students and was comparing it to states where all students took the SAT. One of my early lessons in making sure that apples are being compared to apples.

The report that Thompson discusses doesn't answer all the questions but it at least sheds light on some of the underlying issues of race, emigrant status, class, income, etc. that hinder like-to-like comparisons.