Sociologies uncomfortable relationship with robust metrics, measurements, data, and predictions

Yesterday I saw a series of tweets from a researcher Zach Goldberg posted some data that seemed to be significant about a significant topic in the field of psychology.

More evidence that the popular 'racial resentment' scale is more a measure of intersectional wokeness than anti-black animus

(note the only groups that reach negative ratings in the entire graph) pic.twitter.com/6H7ALCcYw0

— Zach Goldberg (@ZachG932) January 30, 2020

It looked intriguing but I really could not make sense of what measures were being used and what the data was supposed to be telling me.

In the field of psychology measures are often two or three phase shifts of indirectness away from direct measure of real phenomenon. By which I mean there are relatively direct measures of reality. How fast is John Smith running - start line, finish line, stop watch. You have a fairly useful measure of reality. How happy is John Smith - That one is more difficult. What is happy versus content versus serene? What is condition versus perception. Is happiness a context dependent phenomenon or a personality trait? And whatever we define it to be, how do we measure it? Brainwaves? Self-Reports? Blood flow? Serotonin levels? Beta waves? Or something even more indirect? Number of people who want to spend time with the target? Volume of laughter? etc.

So when I saw a complex measurement system that was likely one or two phase shifts away from direct measure, I abandoned consideration of the tweet. I figured that if there was a real there there, someone would eventually intervene and translate.

And now they have. Lee Jussim has brought sense to the incomprehensible representation of measures and data. There is still the issue of indirectness of measurement but that is fundamental to the field. Here is Jussim explanation of what is being represented.

This is an amazing result, substantively and methodologically, that also may be another symptom of political bias in social science. Thread on Racial Resentment as construct and measure. 1/n ending in end. https://t.co/CXndyxPFc0

— Lee Jussim (@PsychRabble) January 30, 2020

That is helpful.

Are the findings of the research real and useful? TBD. I think the value here is more as an example of our frequent overestimation of the pertinence of data. Too often we impute reality, accuracy and precision to information in the form of data simply because it is data. It is hard to keep in mind all the discounting caveats behind the data in order to arrive at a suitably weighted estimation as to whether the data is actually telling us something that is usefully true.

In order to spare reading the tweet stream what this data suggests is that the ideological conviction that there is institutional racism everywhere is at least wrong. Or at least questionable.

My take on the matter is that the claim is badly framed. Everyone carries a population of stereotypes, tropes and heuristics, which are rarely uniformly true but are frequently usefully true. Racism, sexism, xenophobia, etc. are rarely conditions in and of themselves. The problem is not with the stereotypes, tropes and heuristics. The problem is when people refuse to update those stereotypes, tropes and heuristics when counter-evidence is presented. That then becomes bigotry. And malicious bigotry takes infinite forms, not just race, gender, xenophobia, etc.

Goldberg's data is consistent with that view. He finds that Racial Resentment (a form of measured racism) is only weakly correlated with warmth scale. If there is racism, then Racial Resentment ought to be pretty strongly negatively correlated with the warmth or coldness of feeling towards the target group. If you have a high Racial Resentment towards them, then you also ought to feel pretty cool towards them.

Goldberg's data suggests that there is little correlation between RR and feelings towards black men. The degree of measured Racial Resentment does not predict how warmly or coldly you feel towards black men.

There is a weak negative correlation for black women. Degree of Racial Resentment does provide a weak prediction for decreased warmth.

Maybe the measures are simply not good reflections of reality but this is a surprising result and suggests that the current measures of Racial Resentment are not actually capturing actual racial resentment, or that there is no link between Racial Resentment and manifested behavior. Both are possible.

As an example of the latter, I know plenty of people who are philosophically opposed towards open borders immigration who also have extensive networks of warm relations with both foreigners and fellow Americans of different cultural backgrounds. That would be a case where imputed Xenophobic Resentment is not correlated with warmth towards foreigners, or is correlated in the opposite direction anticipated. That isn't uncommon. Opposition to immigration can be construed as xenophobia but it can also originate on perfectly valid economic, sociological, and philosophical grounds as well.

But Goldberg's data throws up something else which is as perplexing. From Jussim:

The only groups to which people are openly negative (Thermomometer scores<50, y-axis) are White men and women; and the only people hostile to them? Are those with the LEAST "anti-black racial resentment" (low scores on x-axis). Claiming on qaires variations of "yeah, blacks are massively oppressed and are not responsible for their lives and have gotten an unfair shake" does not predict warmth towards blacks as much as it does hostility towards whites, especially rich ones. [snip] But Zach's discovery that the most powerful thing RR scores seem to predict with respect to feeling thermometers is *attitudes towards to rich whites* wherein LOW RR scores correspond to hostility to rich whites, is a douzy.

Low racial resentment scores do not predict low warmth towards blacks. But low racial resentment scores do predict high antipathy towards whites is a stark surprise.

It is perfectly consistent with the positions of much of the hard postmodernist intersectionalist left where a love of diversity is highly correlated with antipathy towards whites and Asian-Americans is a common condition.

Goldberg's data is interesting but it is just one more step towards improving actual understanding. The measures are too indirect and unreliable and the whole field is ideologically motivated. But his data does serve to profile just how weak our measurement systems are and the need for a much stronger philosophical and empirical foundation.