Perry Program and life outcomes

I am not surprised, but this is disappointing. From Preschool Stories: The results of the legendary experimental Perry program don’t live up to the hype. by Kay S. Hymowitz.

The Perry program was a tiny experiment in pre-school education from fifty years ago but because it is one of the few programs to have been rigorously tested AND because it showed some longterm positive outcomes, it is the single most relied upon evidence to support pre-school education.

I do not at all object to pre-school education. I object to wasting money on programs that do not achieve their stated goals or at least pay for themselves. Virtually all scaled pre-school education programs fail to achieve their goals and fail so badly that they do not pay for themselves. That doesn't mean we abandon a commitment to our children and to our disadvantaged. To me, it means that we go back to the drawing board and figure what does work and quit wasting money on what doesn't work.

Hymowitz turns up additional information about the longer term outcomes of the Perry program, of which I was unaware. Most of the assessments of Perry Program outcomes stop when the participants were adults of 27. What Hymowitz reveals is that there was an age 40 assessment which calls into question the overall outcomes.

Almost all pre-school programs show at least some improvements in child behavior and achievement in the first couple of years. All show declines after a couple of years and the test and control groups converge very quickly so that it is no longer possible to discern any difference between the two in terms of outcomes. Perry is one of the few programs which showed long term positive outcomes. From Hymowitz's article:

It’s not unusual for decent preschools to produce temporary improvements with disadvantaged students, and Perry did. The treatment group achieved higher standardized test and vocabulary scores than the nontreatment control group, though the differences faded by fourth grade, a common finding in the preschool literature. What set Perry apart were its apparent lifetime effects. Eighty-eight percent of the Perry girls (the girls in the treatment group, not the nontreatment control group) earned a high school degree, compared with only 46 percent of nontreatment counterparts. As teenagers, Perry girls were significantly less likely than their nontreatment counterparts to get pregnant. The Perry boys didn’t show the same dramatic education advantage as the girls: 54 percent of the male treatment group graduated high school, compared with only 43 percent of the nontreatment group. Perry boys also committed fewer crimes than the boys in the nontreatment group. Between reduced prison and victimization costs, lower crime rates created by far the biggest cost-benefit of the program.

There were other gains as well. By the time they were 40, compared with their nontreatment peers, Perry girls and boys earned more money and were more likely to own their own homes, have at least one car, open a savings account, get married, and report fewer health problems. One can see why policymakers got excited by these results.

Sounds good, right? But . . .

There's always a but.

The information of which I was unaware casts even these outcomes into a different context.

But—and you will almost never see this caveat in the thousands of celebratory references to the program—the best that can be said about the Perry kids is that they wound up less poor than their untreated peers. At age 40, those in the treatment group had median annual earnings (including benefits) of $20,800 (in 2000 dollars), versus $15,300 for the untreated. That’s a substantial difference but not enough to take the Perry grads out of the ranks of the “near-poor,” as conventionally defined. (The poverty threshold for a family of four in 2000 was $17,603; “near-poverty” is defined as between 100 percent and 125 percent of the poverty line.)

Nor is that the only reason to temper Perry-based hopes. The nontreatment-group women, who were more likely to become teen mothers, had higher rates of welfare dependency before age 26. But there were no significant differences in the overall nonmarital birthrates between the two groups—and by 40, female Perry grads, who tended to become mothers in their later twenties, surpassed their nontreatment peers in welfare dependency and nonmarital childbearing. Half of treatment-group women were on the dole at some point between ages 26 and 40, compared with only 41 percent of the untreated. They also clocked far more time on welfare—an average of 59 months—compared with 24 months for nontreatment females. More disappointing still were the findings related to children of the Perry participants. As the report puts it: “The two oldest children raised by program-group members did not differ significantly from the two oldest children raised by no-program group members in education, employment, arrests, or welfare status.” In the debate over economic and cultural causes for poverty, the economic camp argues that if poor parents had higher earnings and savings, less material hardship, more residential stability, and less parental incarceration, their children would be more successful. The Perry data fail to support that theory.

More of the female Perry participants ended up using welfare than the untreated group. Not only did they use welfare more but they used more of it (more than twice as much). Perry participants ended up earning more than the untreated group, but not much more. It is not all negative. The male participants still committed fewer crimes than their untreated peers.

So the long term results were nowhere near the benefits trumpeted in the near term. And it is worth remembering - there were only 60 some children in the treatment group. When your sample is so small, it doesn't take much to make the results unrepresentative.

Take as an example the finding:

Eighty-eight percent of the Perry girls (the girls in the treatment group, not the nontreatment control group) earned a high school degree, compared with only 46 percent of nontreatment counterparts.

There were about 60 Perry participants. Say half were girls, that's 30. 26 of those graduated from high school compared to 14 in the untreated group. The extra 12 girls graduating is an achievement. But those numbers are really small so only two or three little random variables can have a disproportionate outcome on the overall results.

The other thing worth remembering is that this small experiment was extraordinarily expensive on a per child basis. In 2016 dollars, the program cost about $20,000 per treated child.

And as Hymowitz points out, getting out of the weeds of crime, and out-of-wedlock children, and home ownership, and drug dependency, and welfare usage, etc. the Perry program made little substantive change to the overall group life outcomes. It possibly made a difference for individuals in the group but it made little discernible difference to the group outcomes.

Back to the drawing board.

UPDATE: On a second re-reading, I just caught that caveat "median annual earnings (including benefits)." Since overall income of $20,800 includes benefits, that means that the productivity increase owing to the effects of Perry Program were even lower. I don't have the data ready to hand, but my recollection is that benefits (welfare, income transfers, non-cash transfers, etc.) constitute a very material portion of annual earnings among the very poor. Perhaps something like 30-40%. By including benefits in the base of calculations, the Perry Program life outcomes on the individual are overstated.