Context and effect size

I nearly fell for it. From School Vouchers Help Low-income Minority Students Earn a College Degree by Doug Gavel. I am inclined to believe that increasing choice is almost always beneficial to customers and citizens, partly because it does give the consumer more choice but also because it serves as an antidote to the sclerosis, rent seeking, and regulatory capture which set in when there is no competition and little choice.

The findings.

A break early in life can result in large gains later on. Certainly that is the case for many low-income minority students in New York City. A new study conducted by Matthew M. Chingos of the Brookings Institution and Paul E. Peterson of the Harvard Kennedy School finds that minority students who received a school voucher to attend private elementary schools in 1997 were, as of 2013, 10 percent more likely to enroll in college and 35 percent more likely than their peers in public school to obtain a bachelor’s degree.

10% and 35%? Wow. Those are great numbers. I was about to skim over to the next article when I scanned down further in the article . I was looking for the absolute effect size. If the intervention raised college graduation rates from 60% to 81% (a 35% increase), then that is likely well-worthwhile.

Presumably there was a cost to the voucher program and that cost has to be set against the benefits derived from the program.

So what was the base performance which improved 35%?

Bachelor’s degree attainment was 9 percent for the minority members of the control group; it increased to 12 percentage points among those who used a voucher, an increment of 35 percent.

Good for the one's who did graduate. However, an increase of 3% points off a small base isn't actually moving the dial all that much. 35% improvement sounds great but it is really just an incremental step and likely one whose costs are not covered by the improved outcomes achieved.

Too bad.