The bitter barrenness of Education Research

From What Do We Do with Education Research? by Freddie deBoer.

We have dramatically increased societal investment in public education over the years, both K-12 and higher education with relatively little to show for it.  Plenty of people get a good education and make good use of that education.  But in aggregate we are spending a lot with relatively little to show for it.  Why?

Lots of reasons of course but one of the reasons is simply that the education research field is inundated all the time with hot new topics which are then researched with extraordinarily low academic discipline.  Virtually all fads of the past few years have come up empty whenever a pre-registered, high discipline research protocol is in place.  There is far more passion than there is results.

I was talking to him about all the ed research I was reading and he said, with his usual tact, “I don’t know why you bother.” He went on to say that he thought education research was the type that should largely be abandoned in favor of anecdote and lore-driven practitioner knowledge, with some major exceptions, because the basic empirical landscape was impossible: education research typically involves small effects, big variances, notoriously multivariate research conditions (this is a field where you can literally be confounded by the air conditioning), involving convenience samples or otherwise presorted groupings of no or dubious randomization. My appeals to techniques like multilevel modeling were waved away; there was no way to rescue the signal from all the noise. At the time the p-value crisis was hot in the popular press, and when he said that ed research seemed almost custom-built to produce those kinds of flawed findings, it was hard to disagree.

deBoer wants education research to be effective but is compelled to deal with the fact that it is a meager field in terms of research rigor.

He lists the categories of issue:

• Methodological and data issues. 
• Publication and replication issues
• Conflicting results facilitate selective reading
• Institutional capture and optimism bias
• Accurately measured but controversial conclusions

What to do?  Freddie has suggestions but the probabilities of being adopted are between slim and none.  I did like, however:

Tell the truth. Male students, not female, now need special programs to facilitate their learning and the benefits of affirmative action. The SAT is not an income test. Charter schools do not result in meaningful learning gains compared to traditional publics. “School quality” has no impact on student performance. The racial achievement gap is not merely a wealth gap. Students who perform poorly early in life are overwhelmingly likely to perform poorly later in life. Etc. Each of these is, to some degree and to some people, a statement of profound controversy, primarily because they are seen inconvenient to the political projects that spring up around education. I am not hypocrite enough to say that we know all of these things to be true. (I am still working on this epistemological humility thing.) But I would say that the preponderance of the evidence is quite clear in each instance. The question is, are people willing to accept conclusions that cut against their social and political desires, especially the bipartisan commitment to pretending that there’s some magic bullet that will someday solve our education problems? Based on my experience of the education research and policy world, the answer is no. But we must try anyway.

 No, these real issues are too controversial to deal with in educational academia.  As I have mentioned elsewhere, IQ, cultural norms and (related) familial structure are profoundly predictive of student outcome.  IQ is a biologically determined and cultural norms and familial structure are third-rail issues are profoundly ostracized.

Consequently, we keep coming up with exotic new explanations and programs which never pan out once we apply rigorous research.  Meanwhile, we ignore the three variables which are known to make a difference.