Feeding stereotype with unreliable data

More cognitive pollution. Report: Women are more likely to have serious mental health problems than men by Lena H. Sun.

This doesn't even pass the sniff test and yet here it is in a major newspaper. If men are several times more likely to commit suicide, a gross proxy for severe mental health issues, then how can it be that women have more serious mental health problems? There actually are some possible scenarios where that circle can be squared, but not obvious ones.

So right away, there should be suspicion.

Here's the lede:

Women in every age group in the United States were more likely than men to have serious mental health problems, according to federal health statistics released Thursday.

The report from the National Center for Health Statistics, part of the Centers for Disease Control and Prevention, also found that more than one-fourth of people age 65 or older who are afflicted with these mental health problems have difficulty feeding, bathing and dressing themselves.

People with serious psychological distress are also at greater risk for certain medical conditions: four times as likely to have COPD (chronic obstructive pulmonary disease) as men and women without mental health problem and twice as likely to have heart disease or diabetes.

OK, more warning signals. What's the implied connection between sever mental illness and COPD, heart disease and diabetes? Again, it is possible to come up with some possible scenarios, but you have to work hard at it. Three paragraphs in and this is beginning to feel like a data fishing expedition. They had a large data set and they sought unexpected correlations without controlling for confounding variables or context.

Pratt said she could not explain why women have higher rates of serious psychological distress. “As I’m sure you are aware, we see this in major depression as well, but I don’t know that anyone has ever come up with a definitive answer of why that is,” she said.

Is "serious psychological distress" the same thing as "severe mental illness"? It doesn't seem like that to me but that appears what they mean in the article.

Ten paragraphs in, and the final paragraph, we begin to get to the root of the cognitive pollution.

The data from the report come from face-to-face interviews conducted by the U.S. Census Bureau, collected for the National Health Interview Survey, a continuous survey with results released annually. Part of the survey includes six questions used to identify people with high likelihood of having a diagnosable mental illness. It captures information about people with serious mental health problems, but it doesn't provide enough information to make a diagnosis as to what specific mental illness a person has.

So this is really all self-reports. Yes, a large data population (good), but it is all self-reported data (bad), and apparently self-defined data (really bad).

Without some objective measure of whether what is reported is actually true, all we can really say, contra Sun's headline, is that "Women are more likely to report serious mental health problems than men." That's a whole different kettle of fish.

Why is this important? I would argue that at least in part it is important because there is an age-old trope of women being hysterical and unbalanced. This self-reported data fuels that negative stereotype. If you are going to reinforce a negative stereotype, I wish the Washington Post or Sun would at least check to see if the reinforcing data is actually reliable or not. Feeding the trolls with weak data for clickbait is unworthy.