News consumption and bubbles - contra evidence

From The Problem With Facts by Tim Harford, a typical rationalist defense of the status quo and the established elites. Which sounds perhaps harsher than I quite mean.

There is a strong strain of establishment characters who really wish people would simply accept the truth of the world as they see it. The problem is that as information becomes more accessible, more people feel emboldened, with and without credentials, to challenge the accepted narrative. Sometimes those challenges are poorly founded, sometimes they drive towards additional insight.

Those who are at the top of the power pyramid are always incented to protect their status and enviable position, and dislike having to justify that position by engaging in debate with those whom they consider uninformed.

Harford is laying out the usual rationalist argument that there are facts and those facts are under assault by ignorami and by those who seek to benefit by obfuscation and misdirection. All of which is true enough. The blind spot is that rationalists, especially establishment rationalists do the exact same thing, usually without acknowledging it.

Put somewhat differently, establishment rationalists often wish to expand the definition of FACT to include everything they and their ilk believe, without being open to debating those facts or even acknowledging doubt. There is plenty that we accept as fact which is really just usefully true.

For example, in the sixties and seventies, it was accepted as usefully true by most international development types that undeveloped countries were poor because they lacked resources and that was used as the justification for massive and longstanding transfers of money between countries. Failed states do not fail because they lack resources (see Venezuela, Russia, etc.). They fail for a variety of reasons, usually having to do more with culture, institutions, exogenous events, etc..

Accepting the kind of standard Harford is pushing is what kept us pushing foreign aid. Aid which was often counterproductive and even destructive to the receiving nation. "Poor countries are poor because of an absence of resources" was exactly the sort of received wisdom which we still indulge; believed to be true but not true. Those that argued against foreign aid because it subverted development or corrupted institutions were dismissed then, just as critics are similarly dismissed now, as uninformed and heartless.

All of which is editorializing on my part. I agree with Harford that we need better decision making and more facts, I am just skeptical that people are 1) willing to acknowledge how little knowledge is necessary to reach useful decisions and 2) how much information we blithely accept as fact which are really simply assumptions yet to be robustly tested.

What did catch my attention was this paragraph.

Last year, three researchers — Seth Flaxman, Sharad Goel and Justin Rao — published a study of how people read news online. The study was, on the face of it, an inquiry into the polarisation of news sources. The researchers began with data from 1.2 million internet users but ended up examining only 50,000. Why? Because only 4 per cent of the sample read enough serious news to be worth including in such a study. (The hurdle was 10 articles and two opinion pieces over three months.) Many commentators worry that we’re segregating ourselves in ideological bubbles, exposed only to the views of those who think the same way we do. There’s something in that concern. But for 96 per cent of these web surfers the bubble that mattered wasn’t liberal or conservative, it was: “Don’t bother with the news.”

Woof! Only 4% were informed enough to be the subject of study on a topic? I fully accept highly skewed Pareto distributions, but 4%. That's pretty extreme.

Here is the original paper that was the source of the 4% claim; Filter Bubbles, Echo Chambers, and Online News Consumption by Seth Flaxman, Sharad Goel, and Justin M. Rao. The abstract:

Online publishing, social networks, and web search have dramatically lowered the costs of producing, distributing, and discovering news articles. Some scholars argue that such technological changes increase exposure to diverse perspectives, while others worry that they increase ideological segregation. We address the issue by examining webbrowsing histories for 50,000 US-located users who regularly read online news. We find that social networks and search engines are associated with an increase in the mean ideological distance between individuals. However, somewhat counterintuitively, these same channels also are associated with an increase in an individual’s exposure to material from his or her less preferred side of the political spectrum. Finally, the vast majority of online news consumption is accounted for by individuals simply visiting the home pages of their favorite, typically mainstream, news outlets, tempering the consequences—both positive and negative—of recent technological changes. We thus uncover evidence for both sides of the debate, while also finding that the magnitude of the effects is relatively modest.

But what does that 4% really mean?

As discussed above, for theoretical and methodological reasons we limit our analysis to individuals who regularly read online news. Specifically, our primary analysis is based on the subset of users who have read at least ten substantive news articles and at least two opinion pieces in the three-month time frame we consider. This first requirement reduces our initial sample of 1.2 million individuals to 173,450 (14 percent of the total); the second requirement further reduces the sample to 50,383 (4 percent of the total). These numbers are generally lower than past estimates, likely because of our focus on substantive news and opinion (which excludes sports, entertainment, and other soft news), and our explicit activity measures (as opposed to self-reports).

OK. That means that only 4% of the population in a three-month period have read at least ten substantive news articles and two opinion pieces. YIKES!

OK, I talk about bubbles all the time but that data shows how much a bubble I am in. I read ten substantive articles and a couple of opinion pieces in a morning. And then again in the afternoon. Every day.

We do have to take Flaxman et al's numbers with a grain of salt. They clearly are making the best of the data they have but they identify and discuss multiple instances where they have to accept proxies owing to the absence of direct empirical data. I think they are on the up and up but the large number of proxies makes for a big asterisk.

Their description of this 4% finding is a mastery of understatement.

It is perhaps surprising that such a small fraction of the population regularly reads online news, particularly opinion articles. While many individuals presumably learn about current events through alternative sources, such as television, radio, or word of mouth, it is likely that many others are simply not politically engaged. Whatever the ultimate cause, the fact that such a small fraction of the population regularly reads online news arguably has considerable impact on both political discourse and media production, though we by and large leave further discussion of these topics to future work.

I found this piece of information very interesting:

Click to enlarge.

Their conclusion after all the number crunching:

Returning to our opening question—the effect of recent technological changes on ideological segregation—there are two competing theories. Some authors have argued that such changes would lead to “echo chambers” and “filter bubbles,” while others predicted that these technologies would increase exposure to diverse perspectives. We addressed the issue directly by conducting a large-scale study of online news consumption. We showed that articles found via social media or web-search engines are indeed associated with higher ideological segregation than those an individual reads by directly visiting news sites. However, we also found, somewhat counterintuitively, that these channels are associated with greater exposure to opposing perspectives. Finally, we showed that the vast majority of online news consumption mimicked traditional offline reading habits, with individuals directly visiting the home pages of their favorite, typically mainstream, news outlets. We thus uncovered evidence for both sides of the debate, while also finding that the magnitude of the effects is relatively modest.

Despite the opening of access to enormous volumes of information to everyone at little or no cost, only 4% of the population reads much of anything substantive in a three-month period and within that 4%, while there is a segregating pattern (people go to news sources conducive to their pre-existing beliefs), their heavy reading habits outweigh the skewing (by reading what they are already oriented towards, they still get exposed to the other position) so that there is actually only a small effect size in terms of reading bias.

I have always accepted the hypothesis that people live in bubbles, primarily based on education attainment, income and class, and that their views and interpretations are strongly shaped by those bubbles. Nothing in this paper disproves that supposition but it does raise a caution. I would have assumed that patterns of news investigation might also be reasonably correlated with education attainment, income, and class. These results suggest otherwise.

I suppose what this might be suggesting is that knowledge and opinions might be more strongly shaped by accumulated experiences than by absorption of the written word.