Why is there a disconnect between polled public opinion and actual public policy?

From If polls say people want gun control, why doesn’t Congress just pass it? from The Associated Press. While this interview is cast in terms of gun control, it addresses a frequent issue - Why is there is disconnect between what a poll indicates about public support for a policy issue and what is seen in terms of actual legislation and enforcement of public policy? The interview is with a pollster who is also an academic.

This is a fairly important issue. Advocates making arguments frequently cite poll numbers to bolster their position. Poll numbers also influence people's estimate of what other people think. Beyond that, polling can be an important empirical signal but virtually everyone, including legislators, judges, academics, researchers, pundits, politicians, the public, etc. dramatically misinterpret what polls can actually tell us.

Gun control is not by any means, the only issue in which there is a disconnect between what polls tells us the public wants and what legislators are willing to enact and the executive willing to enforce. Capital punishment is another public policy issue where there tends to be a sizable gap between the public and their legislative representatives. It is rare that legislators follow the public opinion, they lead. In most OECD countries, at the time that legislators eliminate capital punishment, there is usually 60-80% public support for capital punishment. In other words, legislators are imposing a policy against the will of the people. It is not uncommon for their to be a 20-40 year lag before public support for capital punishment falls below 50% and even then, there remains a strong plurality support for it. It is another example of that disconnect between public desires and actual legislation, the public sometimes leading and sometimes lagging legislation.

From the interview with Harry L. Wilson.

After the recent deadly shooting at a Florida high school, many Americans are asking that question about the federal government’s firearms policy. Recent polls show that a majority of Americans support stronger gun laws – including tighter restrictions on purchasing and a ban on assault weapons – in the wake of the shooting.

Students demand that elected officials “do something,” and many adults echo that sentiment.

But policy does not always follow public opinion. Why are the public’s pleas on this and other issues ignored?

I am a pollster and a political scientist who studies gun control. I have examined the issue from different perspectives.

I have found that there are three major reasons that policy does not always follow public opinion: the structure of the U.S. government, the overlooked complexities of public opinion and the influence of voters and interest groups.

Wilson goes on to make a number of points, but all in the context of gun control. I want to explore the larger issue of the disconnect between polls and policy. It is not nearly as tidy as Wilson's three major reasons, but here are thirteen reasons I think polls do not correlate with actual policy.

Fantasy choices - Many polling questions are about propositions independent of feasibility or consequences. They invite affiliative responses rather than reasoned responses. Open borders, balanced budgets, zero defense spending, equal and excellent education outcomes, free healthcare, secure retirement incomes, cheap mass transit used by everyone, drug free society, comprehensive mental healthcare, affordable housing for everyone, low taxes, etc. All these are potentially desirable outcomes but each of them face perhaps insurmountable obstacles in reality. It is easy to say yes to that which is desirable but which might not be attainable. As long as the poll respondent does not have to bear the cost or consequences of the proposition, there is a strong bias to answer yes to the question.

Brings to mind the old Camelot musical:

It's true! It's true! The crown has made it clear.
The climate must be perfect all the year.

A law was made a distant moon ago here:
July and August cannot be too hot.
And there's a legal limit to the snow here
In Camelot.
The winter is forbidden till December
And exits March the second on the dot.
By order, summer lingers through September
In Camelot.
Camelot! Camelot!
I know it sounds a bit bizarre,
But in Camelot, Camelot
That's how conditions are.
The rain may never fall till after sundown.
By eight, the morning fog must disappear.
In short, there's simply not
A more congenial spot
For happily-ever-aftering than here
In Camelot.

Single issue versus portfolio polls - Very much related to fantasy choices. Free healthcare is theoretically desirable, and if asked if they want free healthcare, people will, given a fantasy choice, answer yes. But of course healthcare costs, it is only a matter of who pays. If the pollers ask a portfolio of questions, free healthcare, low taxes, balanced budget, etc. the obvious issue of contradictions between those public policy is foregrounded and people become more selective in providing affirmative responses. Support for free healthcare declines because people are made more aware of contradictions and trade-offs among a range of choices.

Decontextualization - Polls overwhelmingly go for clarity over nuance. Context is stripped from the issue and one public policy is made to fit all. If asked whether excellence in public school education should be a top priority, most people are likely inclined to answer yes. But the question lacks context. How much will it cost? Who will benefit, in what ways, and by how much? Who will suffer, in what ways, and by how much? Which policies? How long will it take? What are the second order impacts? What other public policies might be affected and in what way? What other personal goals will be affected and in what way? Is the respondent taking into account long term strategic considerations as well as short term tactical considerations? These are all contextual issues that need to be understood in order to comprehend what question the poll respondent is actually answering. Polls are designed to be answered easily and clearly and therefore the questions are stripped of all context. Without context, the answers become near meaningless.

We are a Republic and not a direct democracy - All polls are essentially an exercise in comparing apples and oranges. Direct democracies are highly unstable and subject to fits of mob rule. The Founding Fathers wanted the benefits that come from democracy (consent of the governed) without the historical instability and risks attendant to direct democracy. Part of their answer was to have a republic (our individual voter desires are represented indirectly through our elected representatives.) A Republican style of government is designed and intended for there to be a gap between fickle public opinion and settled public law and policy.

A poll is an act of direct democracy. It measures the momentary opinion of the public on an issue. Legislation (and execution through the executive branch) is conducted by our elected representatives. There will always be a gap between the two because we are measuring two different things.

We are a federal government and not a parliament/unitary government - A second strategy on the part of the Founding Fathers to constrain mob rule and ensure minority rights and interests was to incorporate three levels of (often competing) government: municipal/county, state and federal. This is most manifest in the Senate where all states have only two representatives regardless of their population. It makes it difficult for a small number of states with large populations to run roughshod over the interests of a large number of states with small populations.

For a general federal election, it is not sufficient to poll a thousand people across the country. That won't tell you much about the likely election results. Because of the Electoral College, there is not a single popular vote, there are fifty state votes. You have to know which way each state is going to swing in order to know the direction of the election.

The general public do not vote, voters vote - Polls measure the sentiment of the general public but representation (and therefore policy) is determined by who actually vote. Pollsters try and address this in general elections by measuring three different results: the general public, registered voters, and likely voters. The numbers are different. When you are dealing with single issue polls, though, it is harder to screen whether you are polling an issue with salience to the respondent and the degree of that salience. The general public may support X. But if they are not a committed single issue voter, that does not tell you much about how that general support translates into legislative support.

Advocate intensity differential - Closely related to the importance of distinguishing voters from non-voters is advocate intensity. Part of this has to do with likelihood of single issue voting. 60% might support some unstated form of gun control but only 5% might be willing to be a single issue voter on the policy. Intensity alone is insufficient to determine outcomes unless that intensity is widespread. Many hot topic public policy issues have wide but shallow support.

The second aspect of advocate intensity has to do with appearances. For example, it appears that university campuses are substantially dominated by social justice warriors seeking to impose their radical will on the general public. But that is simply an appearance. 1-5% of the student population is willing to forego their studies in order to demonstrate, deplatform speakers, shout down speakers, harass their fellow students, etc. They create the impression of broad support for their agenda simply through their extreme commitment. The fact that most students do not share their agenda is hidden behind their noise and notoriety.

Choice constraint - Polls are weak at capturing the disconnects that occur because of choice constraint. As an example, I am a deficit hawk. I am deeply concerned about the lack of attention to the long term strategic dangers of deficit spending for consumption. I might be a single issue voter if I had the opportunity, but elected officials who are deficit hawks are few and far between. Consequently, there is a huge gap between my personal preference for controlling deficit spending and how I actually vote. If a poll asks my opinion on deficit spending, I will affirm a real passion for controlling deficit spending. But that passion will not be reflected in my voting owing to choice constraint. Since there are rarely politicians committed to deficit reduction, I end up voting for those most likely to take an evidence-based approach to spending since that is the closest I can get. If they are going to spend the money anyway, at least do so intelligently.

Revealed public preference paradoxes - The public is not only fickle, but they appear logically incoherent on occasion. Mass transit is a prime example. 74% of the public supports more mass transit spending. When given the opportunity, 80% of mass transit spending ballots pass. But only 5% of commuters choose to use the public transport which they support and finance. Voters aren't logically incoherent, they simply want other people to ride mass transit. But that is not how polls are framed. Polls ask, "Do you support public transportation?" and people are answering, "Yes, I support other people riding public transportation?"

Positional progress and ignorance of power laws - This is an obscure one but of reasonable significance. Where the public perceives it is on the S-curve of development and how it factors in the power laws of the S-curve are singificant considerations in public policy but never addressed in polls. It relates to decontextualization. Public healthcare is an example. It very much matters where you are on the healthcare S-curve when you ask about policy. If you are a still developing country, at the bottom of the healthcare S-curve, capital investment in basic health infrastructure such as clean water, pollution control, and basic clinics can have massively disproportionate positive impacts on longevity and morbidity. That is why virtually every poor country in the world today has a higher longevity than the most developed nations did in 1950. If you are at the top of the S-curve, increasing expenditures have dramatically less impact and, indeed, can be inconsequential. See the Oregon public health insurance experiment that showed no results for a change in policy that was expected to have significant improvements. A hotly debated, and polled, public policy where the likely S-curve position was ignored.

Poll design and framing - This is reasonably widely acknowledged but there is a limit to what can be done because informative polls would be extraordinarily expensive. Cheap, fast and easy dominate but are close to meaningless in many situations. Push polls (framed in a way to deliver a desired poll outcome) are frequent and muddy the waters. But even well intentioned polls are subject to faulty design, false positives, exogenous events (a terror attack during a weeklong polling on terror will skew the results for example), sample sizes, non-randomization of the sample, sample salience errors, etc. Sample size is an endemic issue, exacerbated by the federal republic issues discussed earlier. A random sample of 1,000 people should generally be sufficient to get a moment in time read on the public (subject to all the caveats above.) For a single question at a national level. But journalists and analysts rarely stop there. They then want to know what the results are by gender, race, state, income level, partisan affiliation, education attainment, etc. It is not unreasonable to want to know what college educated African Americans in California think about policy X compared to high school educated white Americans in the Midwest. Reasonable, but the poll is not designed to answer that degree of particularity. If it is truly random, then the response average is based on roughly 7 people in the first instance and 122 people in the second. Those numbers are dramatically too low to give you any statistical credence.

Goal prioritization differential - This relates to single issue and decontextualization. Every person has a range of issues which they regard as pertinent to them and they prioritize those in some fashion. The economy usually ranks very high as does personal security. International issues are usually low. Social justice issues are vanishingly small. So everyone may be interested in and, to some degree, committed to good public education (as an example.) But that tells us nothing. We need to know the degree of interest. If one person answers that it is their number one priority from a list of twenty and one person says it is number twenty, then there is an important signal that is overlooked. The poll may show that 100% are interested in better public schools but we need to know the relative ranking of that goal. If it is the number one goal for 100% of respondents, that is different than if it is number one for 5% and number 20 for 95%.

Goal trade-off profiles - This relates to goal prioritization. Not only do we need to know the relative ranking but we need to understand the respondent's perceived trade-off weightings among those twenty goals. The single issue voter might have education as number one in ordinal ranking and will give 81 importance points to education and weight the other nineteen issues as tied with one point each. They are important but deeply discounted against education. More typically, the next respondent might have education in fourth position and assign it only five points. They might rank the economy as first with forty points, employment as second with twenty points and the environment as third with ten points. The trade-off profiles are dramatically different even though they all agree on the importance of education. Trade-off profiles can make single issue polls very misleading.

To the degree that any or all of this is true, which I think it is, then that prompts a different question. If polls are so fallible, why do we accord them so much importance?