Cheap and easy propositions need some hard facts

From ACT/SAT for all: A cheap, effective way to narrow income gaps in college by Susan M. Dynarski. I am deeply skeptical of all silver bullet solutions based on a fairly extensive consistent pattern of over-claiming beneficial outcomes, under-estimating costs, risks and unintended consequences, and a near perfect record of failing to deliver the promised outcomes.

But this looks promising.

There are many logistical hurdles on the road to college: financial aid forms, admissions essays, letters of recommendation, and entrance exams. There are dozens of details to remember, deadlines to meet, forms to complete, and fees to pay. Parents who have gone through this themselves, and have the time and resources, can coach their children through this process. Other kids are largely on their own.

These seemingly minor obstacles put many low-income students off the path to college. A study of high school seniors in Boston found that few low-income youth “decide” against college. Rather, they miss a key deadline, or incorrectly fill out a form, or fail to take a required class, and thereby fall off the path to college.

Consider the ACT and SAT. These entrance exams are required for admission to virtually all selective colleges in the US. Students have to register and pay for these tests, and then travel to a testing center on a weekend to take them. This is straightforward, if you have internet access, a computer, a credit card, and a car. If you are missing any of these resources, it’s a lot more challenging. The nearest testing center may be in a suburb that is unreachable by public transportation early on a Saturday morning.

But, in a dozen states, the ACT or SAT is now given in school, for free, on a school day during school hours. In most cases, the ACT or SAT replaces the standardized test that students would otherwise take in high school, so there is no additional time spent testing. This is an attractive feature, given the widespread backlash against perceived over-testing in schools. Sitting for the test is also required, which means that students can’t opt out because of low expectations – whether theirs or those of the adults around them.

In Michigan, in 2007, the ACT became part of the test required of juniors in the public schools. As a result of this shift in policy, the share of Michigan’s high school students taking a college entrance exam rose from 54 percent to nearly 99 percent. The growth was even sharper among low-income students, of whom only 35 percent were previously taking the test.

In a globally competitive economic system, an aging population and an increasingly complex world, we need all the talent we can find. And it is morally distressing that some might be being left behind because of low expectations. All of which makes this initiative sound extremely attractive.

One concern is that the report is squirrely in the reporting of the results.

The results were surprising. Thousands of academically talented students in Michigan had not been taking the ACT (or the SAT, which Hyman also tracked). For every 1,000 students who scored high enough to attend a selective college before testing was universal, another 230 high scorers were revealed by the new policy. Among low-income students, the effect was even more dramatic: for every 1,000 low-income students who had taken the test before 2007 and scored well, another 480 college-ready, low-income students were uncovered by the universal test.

What's that mean? I don't want relative rates. I want some sort of outcomes.

Did the number of students with a very high score of, say 1400, go from 100 students to 150 students. Or did it go from 100 to 110? It is impossible to see the whole picture based on what they are reporting.

And independent of how they scored, how many of those with a score 1400 actually enroll in university? Did it go from 100% of 100 to 100% of 150? Or is it 92% of 110?

And independent of how many actually enrolled in university, how many actually graduate? In other programs to increase the number of low income students at high reputation universities, the dropout rate is notoriously high.

I am concerned that Dynarski reports on none of this. Her idea sounds simple and promising, but the truth is in the numbers. But if these programs have been going on for a decade at some of these states, why aren't they reporting these results? The promise of a cheap and easy silver bullet without definitive results matches too closely the results of most such efforts over the past couple of decades. Efforts which ended expensively in failure.

And for all the chatter that this is cheap and easy to do, it is not free. There is a cost. In order to know whether it is worthwhile, we need to know the numbers to the above questions.