Tuesday, August 12, 2025

The Null Hypothesis as a default forecast for all centralized initiatives in complex systems.

From Why isn't every child using Mastery Learning? by Arnold Kling.  The subheading is Are schools leaving $20 bills on the sidewalk?

I believe what I call The Null Hypothesis. That is, very few education interventions can be proven to work by using a controlled experiment. Of those, very have effects that do not “fade out” with time. Of those, very few can be replicated outside of the original setting.

I would make explicit one further consideration.
  1. Few interventions can be proven to work with a controlled experiment.
  2. Most effects fade relatively quickly.
  3. Most cannot be replicated by independent parties.
  4. For most, the costs outweigh the benefits.
I agree with Kling and extend this to any institutionally driven intervention.  If it is not self-generated emergent order, then the Null Hypothesis is a good first approximation of probable effectiveness.  

The Null Hypothesis is likely to usefully describe anything from Academia, from NGOs, from advocacy groups, from theology schools, and from institutions. 

See the Millennium Villages Project for a quintessential example of an idea from academia and advocacy groups which fits the Null Hypothesis quite neatly.


No comments:

Post a Comment