Everybody cares but nobody cares enough to do it right.

From Nobody Knows How Well Homework Works by Scott Alexander. 

A decade or so ago, I owned a small business, Through the Magic Door, whose mission was to help parents create an environment where their children were more likely to become habitual readers.  The mechanism was basically to match children's reading interests to the titles available with a particular focus on those books which had stood the test of time.  Children who enjoy what they are reading are more likely to keep reading.  

I learned a lot out of that business.  One piece of information was just how little most people read and how much reading is dominated by a small population.  From recollection, I think perhaps 50% of adults read no books electively in a year.  The old stalwart Pareto Principle was alive and well 80% of books purchased were purchased by 20% of readers.  And while there were some variations by country, these figures were largely mirrored across the OECD.

There were interesting insights such as most Young Adult literature is actually read by adult women.

And then there was this issue that Alexander is writing about.  Nothing has changed since I looked at it ten or fifteen years ago.  

Despite its centrality to some tens of millions of children in this nation alone and for the entirety of universal public education (say, a century), no one has a mathematical model connecting amount of time spent on homework with difficulty of assigned homework with quality of educational achievement and attainment.  

The assumption is that more homework, or at least, more hours spent on homework, will yield better educational outcomes.  The child will learn more and remember longer.  

But that has not been demonstrated.  There have been hundreds and thousands of studies but they are quintessential social sciences studies.  No pre-registered methodology, poor definitional control, small population size, no control of confounds, etc.  We have spent decades producing almost worthless studies that reinforce entrenched emotional assumptions rather than demonstrating actual causal relationships.  

And the confounds are legion and easily anticipated.  You have to control for IQ, parental education attainment, class, living conditions, difficulty of material, variability in teacher competency, variability in teacher expectations, variability in teaching methodology, curriculum, etc. 

With tens of millions of teachers and billions of child years over a century, hundreds of teacher degree granting institutions, tens of thousands of studies, and billions in study grants - and we still have no real clue about the relationship between difficulty of homework assignments, quantity of homework assignments, and educational outcomes achieved.  

Astonishing.

Here is an excerpt from Alexander:

But this is the least of our problems. This methodology assumes that time spent on homework is a safe proxy for amount of homework. It isn’t. Students may spend less time on homework because they’re smart, find it easy, and can finish it very quickly. Or they might spend more time on homework because they love learning and care about the subject matter a lot. Or they might spend more time because they’re second-generation Asian immigrants with taskmaster parents who insist on it being perfect. Or they might spend less time because they’re in some kind of horrible living environment not conducive to sitting at a desk quietly. All of these make “time spent doing homework” a poor proxy for “amount of homework that teacher assigned” in a way that directly confounds a homework-test scores correlation. Most studies don’t bother to adjust for these factors. The ones that do choose a few of them haphazardly, make wild guesses about what model to use, and then come up with basically random results.