Centralized collaborative teams are more susceptible to believing claims which are false than are decentralized more competitive teams

From Centralized “big science” communities more likely generate non-replicable results by Valentin Danchev, Andrey Rzhetsky, and James A. Evans. From the abstract.

Growing concern that most published results, including those widely agreed upon, may be false are rarely examined against rapidly expanding research production. Replications have only occurred on small scales due to prohibitive expense and limited professional incentive. We introduce a novel, high-throughput replication strategy aligning 51,292 published claims about drug-gene interactions with high-throughput experiments performed through the NIH LINCS L1000 program. We show (1) that unique claims replicate 19% more frequently than at random, while those widely agreed upon replicate 45% more frequently, manifesting collective correction mechanisms in science; but (2) centralized scientific communities perpetuate claims that are less likely to replicate even if widely agreed upon, demonstrating how centralized, overlapping collaborations weaken collective understanding. Decentralized research communities involve more independent teams and use more diverse methodologies, generating the most robust, replicable results. Our findings highlight the importance of science policies that foster decentralized collaboration to promote robust scientific advance.

That is kind of opaque. The closing discussion is marginally more enlightening.

Extensive (22) and often overlapping scientific collaboration (16), along with cumulative advantage processes that create central, star scientists (19), produce centralized “big science” communities with dense methodological and intellectual dependencies. Such communities tend to publish fragile findings. Our research points to the importance of science policies that foster competition and decentralized collaboration to promote robust and replicable scientific advance. Our analysis demonstrates the utility of large-scale experiments coupled with enriched article content and metadata to diagnose the replicability of published research. Our findings suggest a calculus for evaluating the system-level trade-off between investments in robust, replicable knowledge, which comes at the price of larger, but weaker, preliminary insight.

If I am reading the study correctly, what they are saying is that if you divide findings between unique claims (different from the mainstream) and received wisdom (claims consistent with the consensus of the community), the unique claims are replicated at half the rate of the received wisdom claims. But in either category (unique claims versus received wisdom) the majority (at least 55%) of claims fail to replicate.

Most of what we know is wrong, and the more out of the mainstream our claims, the more likely it is to be wrong.

Their second finding is the one I am interested in. While the above is true, there is a second way of looking at the issue. Which groups are more susceptible to false knowledge, large, centralized, collaborative teams or smaller, decentralized, competitive teams? What they find is that centralized collaborative teams are more susceptible to believing claims which are false than are decentralized more competitive teams.

That is interesting and entirely consistent with classical liberal worldview (Locke, Smith, Mills, Hume, Hayek, etc.). The fact that centralized, collaborative research groups are more susceptible to group think and false knowledge is also entirely consistent with modern economic and political experience.

These findings also suggest an interesting insight to the challenge of fake news (false knowledge) in complex, modern, dynamic societies.

It is a common, and not unfounded, claim that news media are biased. This claim is usually made in political terms (Democrats versus Republicans or left versus right) and that is an interesting to a degree. But I have made the point frequently in earlier posts that I think there is something different going on which drives the perceived partisan split. I have focused on the homogenization of the media industry. The great bulk of reporters are now concentrated in a small number of cities - New York, Washington, D.C., Los Angeles.

Sub offices and stringers in other locations, sure, but not where any decisions are made. These are all high cost, high density, high inequality, Democratic majority cities. Combine that with all reporters and editors now being highly compensated (upper middle class) and all with college and or advanced degrees, and it is easy to see that they live in a world that is distinctly different from that of most Americans. It is easy to see why there is a large divide between the world as seen by an average American and the world as seen by journalists in dense, expensive, unequal, mass transit enabled, highly unequal, democrat dominated cities. They simply live in two different worlds.

There is another factor which I have mentioned in the past but which this research would seem to weight more heavily. In addition to all the above observations being true, it is also the case that the news collection and distribution industry is now far more concentrated than in the past. Five major media organizations account for 80% or more of the media.

Danchev, Rzhetsky, and Evans find that centralized, collaborative, interdependent communities perpetuate incorrect beliefs to a greater extent than do independent, decentralized, competitive organizations. Perhaps the fact that news media is now concentrated in three cities, among five companies, with experientially homogenous employees who work collaboratively with one another, and with shared worldviews might be the greater explanation for why they are so prone to seize on and exaggerate information which is inaccurate and/or untrue. Perhaps it is not so much an issue of partisan bias (an incidental outcome) but at core simply a product of an underlying system dynamic.

Hugely speculative but an interesting implication.