Saturday, September 8, 2018

3.5% of men forgot they got divorced this past year.

An excellent illustration of the dangers of drawing any conclusions based on a data set of self-reports. Setting aside the extremely small portion of LGBT divorces as a possible skewing element, exactly the same number of men and women are involved in a divorce each year. It is tautological.

However, that's is not what is revealed based on surveys of self-reports.



Each year, about 3.5% more women report being divorced than men. Callous and inattentive men might be, but I still don't think that covers forgetting that you were earlier married in the year and now are not.

What this indicates is that there is, for surveys based on self-report, an inherent 3.5% margin of error simply from faulty reporting of even the most obvious events. Imagine the error rates for what you ate at lunch yesterday or whether you took your medicine consistently over the past week.

This is why effect sizes are so important in research reporting. Anything with an effect size of less than five percent is almost not worth discussing.

2 comments:

  1. > an inherent 3.5% margin of error simply from faulty reporting

    No. It might be true if you had only one year, but since you have a near-constant bias over many years, it can't be simple margin-of-error.

    ReplyDelete
  2. You are correct, it is not a margin of error as it is a persistent undercounting. A structural 3.5% error rate, perhaps. This is not dissimilar to the Lizardmen issue of Scott Alexander (http://slatestarcodex.com/2013/04/12/noisy-poll-results-and-reptilian-muslim-climatologists-from-mars/)

    ReplyDelete