If Hayek was a betting man . .

From Scotland’s ‘No’ Vote: A Loss for Pollsters and a Win for Betting Markets by Justin Wolfer.

Thursday’s Scottish referendum was interesting not just for what it said about Britain, but also for what it said about the state of political forecasting. I’m calling it a loss not only for the pro-independence movement — the “No” campaign won 55.3 percent of the vote — but also for the pollsters.

To be fair, I should start by acknowledging that most of the election-eve polls correctly predicted a majority No vote, but they all underestimated the margin, and many missed by quite a lot. The polls were volatile; they often gave conflicting signals; and it took them until the last few weeks to even start to suspect that this would be a close race. The major polls in the past week ranged from a 6-point lead for the Yes vote to a 7-point lead for the No vote.

And this wide range wasn’t because of wild fluctuations in public opinion. It was the result of two surveys that were taken within a day of each other.

The prediction markets, on the other hand, yielded much more reliable forecasts. Despite the demise of Intrade, these markets remain extremely active, and over at Betfair, bettors rated the chances of a No vote at around 80 percent, an estimate that remained remarkably stable over the past week, fluctuating by only a few points.

A very Hayekian, Problem of Knowledge issue. I especially like this observation which I have seen many times.

My own research with Microsoft’s David Rothschild suggests that pollsters could do a better job if they learned from prediction markets. Instead of focusing on whom people say they plan to vote for, ask them instead to focus on who they think will win. Typically, asking people who they think will win yields better forecasts, possibly because it leads them to also reflect on the opinions of those around them, and perhaps also because it may yield more honest answers.

It’s an idea with particular relevance to the case of the Scottish referendum. As Stephen Fisher, an associate professor of political sociology at the University of Oxford, has noted, there is a historical tendency for polling to overstate the likelihood of success of referendums, possibly because we’re more willing to tell pollsters we will vote for change than to actually do so. Such biases are less likely to distort polls that ask people who they think will win. Indeed, in giving their expectations, some respondents may even reflect on whether or not they believe recent polling.

Indicating that it is both a problem of revealed preference as well as a Knowledge Problem.

Revealed preference has to do with the gap between what people say they want and what people reveal they want through their actions. When presented with a proposal out of context, many people will affirm an interest. When they come to vote, they have to consider the proposal in context and might make a different decision given the implied trade-offs and potential unintended consequences.

Take a hypothetical. There have been a rash of break-ins and street assaults. A group of citizens put a proposal on the ballot to double the size of the police force. It marshals a lot of public support because people are concerned about their safety. When it comes to voting day, people are likely to begin to put the proposal in context. Yes I want greater security. BUT: do I want to pay more taxes? Am I confident that more police will reduce crime?, More police probably means more traffic stops and other citations, is that worth it?, Might a greater police presence elevate other civic tensions? etc.

So while people may tell the pollsters that they support more police for more security, when they actually vote, they are likely taking into account an array of other considerations which might change their vote.

Some interesting tactical suggestions in the article for how to poll more accurately. What I find most interesting though is that commercially incented polling (i.e. betting) is so notably more accurate.

There are the Hayekians out there who want to see what motivated self-interest produces in terms of accurate forecasts, and then there are those who believe that if we are simply smart enough, we can prepare better forecasts.