Asymmetrical Category Relevance

There is something very interesting in this framing which I am wrestling to articulate. If I can nail it down, I think it is a class of error which is common in polling.

64% of Americans say the government provides too much help for wealthy people https://t.co/v5iCdzamu9 pic.twitter.com/jcSWkUCjuS

— Pew Research Center (@pewresearch) February 2, 2018

The basic frame is "Should we give more beneficial things to:" and then offers five categories.

Older Poeple
Poor People
Middle Class People
Younger People
Rich People

Their empirical conclusion is that the surveyed do not want to give more beneficial things to Rich People.

Fair enough. Not a surprising result. The rich are already disproportionate beneficiaries, and are usually well positioned to look after themselves. I am not disputing that these empirical numbers represent how most randomly selected people would respond.

The measurement issue in the framing is that four of the categories are different in nature from the fifth. Specifically, everyone, if not now, will be at some point Old. Everyone, if not now, could have a measurable chance of becoming Poor. Everyone, if not now, could have a significant chance of being Middle Class. Everyone, if not now, either was or has Young people in their lives. Of 1,000 people being surveyed, 100% of them are likely in one of these categories, have prospects of being in one of these categories, or have in their immediate networks of friends and family, members of these categories.

Only 29% people think they will ever be a millionaire. Currently, 5% of Americans are millionaires.

Looked at it from this way, the Pew question actually reads differently. In fact it is two different questions.

"Should we give more beneficial things to groups to which you currently belong, have belonged or will belong?" for four questions.

"Should we give more beneficial things to a group to which you do not belong and do not think you will belong?" for one question.

Again, this isn't about whether Pew randomized their target polling group or whether Pew was misleading in their questions or even whether this is an apples-and-oranges question. The issue is that the frame forces a hidden skew on the responses.

The structure is asking people to respond to two different categories: 1) categories which they see as relevant to themselves and 2) categories which they do not see as relevant.

I think it is a reasonable assumption that people will be inclined towards generosity for categories in which they repose and have/can anticipate being relevant to them. And correspondingly people are likely ill-disposed towards generosity for categories that they see as irrelevant to them.

Let's test this with other candidates in the irrelevant categories (groups of people who are small compared to the respondent population and/or whose distinctive attribute does not have material presence in the world of the respondent populations.)

"Should we give more beneficial things to any of the following groups to which you do not belong and do not think you will belong?"

African-Americans

Asian immigrants

Left-handed people

Muslims

Transgender people

Seventh Day Adventists from Germany

Eastern Orthodox Christians

Atheists

Maronites from Lebanon

Red haired people

Small family-owned farms

With this list I have tried to include groups that are not numerically large within the larger population. I have tried to include a range of groups for whom there are likely sympathies (small family-owned farms), indifference (Seventh Day Adventists from Germany) and antagonisms (Atheists).

Regardless of whether the population has a favorable, unfavorable, or indifferent attitude, I suspect that the responses for all these groups would look like the responses for Rich People. Because these are irrelevant attributes to the surveyed population, they would almost certainly be disinclined to extend more beneficial things to them. Not because of attitude towards the group but because of the irrelevance of the group to the surveyed population.

What do we call this phenomenon? I don't know. The Pew survey is framed in such a way that it appears that people favor Middle Class, Old, Young, and Poor and disfavor the Rich. But that is likely not what it is saying. It is saying that the respondents do not wish to extend favor to those in circumstances deemed irrelevant to the respondents.

The results are not due to relative favoring of groups but due to asymmetrical category relevance.

When pollsters do randomized surveys of a population, say on a political issue, they routinely have to normalize the results. Say the populace is equally divided among Democrats, Republicans and Independents. Once they collect the responses, they discover that the surveyed population deviates from the expected distribution of 33/33/33 and is actually 40/30/30. They have to normalize the results so that it reflects the known distribution of 33/33/33.

What the above line of thought implies is that they should also be doing some sort of normalization to address asymmetrical category relevance. Normalization of relevance equality as it were.

Having now seen the issue, it will be interesting to see how many surveys have symmetrical or asymmetrical category relevance. I suspect it is a more common issue than I might have recognized.

Taking asymmetrical category relevance into account might up-end how we understand various issues. Take some distinctly urban concerns such as densification, gentrification, and rent control. Each of those are more or less pertinent to perhaps some 10-30% of the population (people who live in urban centers). When you do a survey that includes people from the entire population, you have an asymmetric category relevance issue that needs to be addressed.

The best example of asymmetric category relevance is in Gallup's tracking of most important problems. The two big, and fairly constant problems are the Economy and the Government. Everybody is affected by the economy and everybody is affected by government. They are both universally relevant.

But after those two universal categories, it plunges markedly. Just about everything else is in the 0-10% category. Virtually all these other categories are strategic/material/important issues but most of them are only relevant to some subsections of the population.

Returning to the original Pew results, you can see how the reported numbers, unadjusted for asymmetrical category relevance, might skew policy thinking. 95% of people are not millionaires and therefore, their indifference to Rich People makes sense. But because 64% think we are already doing too much can lead to an unhealthy position: Rich People are an outgroup to the majority of the population and therefore it is acceptable to shift more the burden of increased taxes and decreased benefits to them and away from the majority. Taken too far, you end up in the position of Connecticut, New Jersey and others where the outgroup of Rich People are so heavily taxed that the departure from the state of even one or two billionaires can throw the state budget into complete disarray. Despite it appearing from the Pew results that a commanding percentage of the population thought that was a good idea.

RELATED:


Double click to enlarge.