I am so delighted that Slate Star Codex, now in the form of Astral Codex Ten, is now back up and running and generating the intriguing and stimulating content of yore.
WebMD, And The Tragedy Of Legible Expertise is well worth a read. It is a great explication of the unexpected consequences of loosely coupled complex systems which, while reasonably optimized individually, can produce unexpected and sometimes negative composite outcomes when loosely coupled.
Some of his observations:
WebMD is the Internet's most important source of medical information. It's also surprisingly useless. Its most famous problem is that whatever your symptoms, it'll tell you that you have cancer. But the closer you look, the more problems you notice. Consider drug side effects.
The issue is that WebMD is subject to many pressures to reduce reputational, financial, and health risks. By doing so, they incidentally over-elaborate and frequently over-emphasize risks to the point that consumers are unable to usefully discern real medical risks from marginal risks.
The essence of Moloch is that if you want to win intense competitions, you have to optimize for winning intense competitions - not for some unrelated thing like giving good medical advice. Google apparently has hard-coded into their search algorithm that WebMD should be on the front page for any medical-related search; I would say they have handily won the intense competition that they're in. They must have placated a wide variety of stakeholders and fought off a wide variety of attackers; each of those victories took a minor change to their medical information or their procedures for producing medical information. Repeat a thousand times, and they're on top of the world, and also every diagnosis is "cancer" and every drug's side effects are "everything".
WebMD is too big, too legitimate, and too canonical to be good.
This is a great insight to loosely coupled systems and the impact of multi trade-off objectives, all of which are good but not all of which are prioritized in the same ordinal fashion among differing people and among the population.
Why do institutions sub-optimize?I can't tell you how many times over the past year all the experts, the CDC, the WHO, the New York Times, et cetera, have said something (or been silent about something in a suggestive way), and then some blogger I trusted said the opposite, and the blogger turned out to be right. I realize this kind of thing is vulnerable to selection bias, but it's been the same couple of bloggers throughout, people who I already trusted and already suspected might be better than the experts in a lot of ways. Zvi Mowshowitz is the first name to come to mind, though there are many others.
There are all sorts of places you could go with this. Maybe expertise is a sham, and a smart guy thinking for five minutes can outdo a decade of working on a PhD. Maybe Joe Biden is an idiot for not appointing Zvi the Secretary of Health. Maybe the whole system is a plot to keep good people down, and we need to burn it down and start over again. Or maybe I'm dumb and biased, and actually the experts are doing much better than Zvi but I'm selectively misinterpreting evidence until I think they aren't.
Probably all of these have a grain of truth in them. But I find myself settling on a different explanation, which is something like this:
When Zvi asserts an opinion, he has only one thing he's optimizing for - being right - and he does it well.
When the Director of the CDC asserts an opinion, she has to optimize for two things - being right, and keeping power. If she doesn't optimize for the second, she gets replaced as CDC Director by someone who does. That means she's trying to solve a harder problem than Zvi is, and it makes sense that sometimes, despite having more resources than Zvi, she does worse at it.
The way I imagine this is that Zvi reads some papers on whether the coronavirus has airborne transmission, sees the direction they're leaning, and announces on his blog that it probably has airborne transmission.
The Director of the CDC reads those same papers. But some important Senator says that if airborne transmission is announced, important industries in his state will go bankrupt. Citizens Against Lockdowns argues that the CDC already screwed up by stressing the later-proven-not-to-exist fomite-based transmission, ignoring the needs of ordinary people in favor of a bias towards imagining hypothetical transmission mechanisms that never materialize; some sympathetic Congressman tells the director that if she makes that same mistake a second time, she's out. One of the papers saying that airborne transmission is impossible comes from Stanford, and the Director owes the dean of Stanford's epidemiology department a favor for helping gather support for one of her policies once. So the Director puts out a press release saying the evidence is not quite strong enough to say airborne transmission definitely happens, and they'll review it further.
I realize it doesn't sound like it, but I'm trying to excuse the CDC here. I'm not just saying they're corrupt. I'm saying they have to deal with the inevitable amount of corruption which it takes to be part of a democratic government, and they're handling it as well as they can under the circumstances.
Expertise isn't a sham. The Director of the CDC could generate opinions as accurate as (or more accurate than) Zvi's, if she wanted to. Maybe she's even doing that internally, when she decides what precautions she and her family should take. Or maybe she isn't; I know a lot of people who have turned into the mask they put on to succeed, just because it's easier that way. The Director may carefully avoid being the kind of person who can generate opinions more accurate than the ones she has to officially endorse; this is probably the best option for her mental health.
Joe Biden can't appoint Zvi as CDC Director, at least not usefully. If Biden appointed Zvi as Director one of three things would happen. One, Zvi would learn to play politics as adroitly as the current Director, and lose his advantage over her. Two, Zvi would offend enough people that they would pressure Biden to fire him. Or three, Zvi would offend people, Biden would offend people by not firing Zvi, and eventually Biden would fall beneath some necessary threshold of support and not be able to be an effective President. I'm not saying that just appointing Zvi would inevitably get Biden impeached. I'm saying Biden has a certain amount of slack, given how many people he needs to keep happy in order to govern effectively, and appointing Zvi as CDC Director would use up so much of that slack that he couldn't do other equally useful things later without becoming ineffective and likely to lose reelection.
Dr. Fauci (and WebMD) are legibly good (or at least legibly okay). They sit on a giant golden throne, with a giant neon arrow pointing to them saying "TRUST THIS GUY". If a random shmuck who doesn't know anything about anything Googles "who should I trust about COVID?", Google will return Dr. Fauci's name. This is a position of great power; Dr. Fauci is able to make decisions that will affect billions of dollars in wealth, Senate seats, Twitter likes, and other extremely valuable resources. Thousands of people who would prefer that they get the dollars and seats and likes will be gunning for him. In order to stay on that throne, Dr. Fauci will need to get and keep lots of powerful allies (plus be the sort of person who thinks in terms of how to get allies rather than being minimaxed for COVID-prediction).This interferes with his COVID predicting ability, but in the current system there’s no alternative. You can't trivially put Zvi on that throne, any more than you could trivially make Zvi benevolent dictator of the world (another job I think he would be good at). One of the big differences between good and bad systems of government is how much they rely on corruption vs. meritocracy in putting people on those thrones, and our system of government is only mediocre. As the saying goes, "there are no First World countries".
Good materials. All are related to certain organizational adages which have to do with the sub-optimization of organizational objectives owing to the conflicts arising from loosely coupled complex systems.
Some examples:
Freddie deBoer - The Iron Law of Institutions is this: “the people who control institutions care first and foremost about their power within the institution rather than the power of the institution itself. Thus, they would rather the institution ‘fail’ while they remain in power within the institution than for the institution to “succeed” if that requires them to lose power within the institution.”
Jerry Pournelle's Iron Law of Bureaucracy
...in any bureaucratic organization there will be two kinds of people: those who work to further the actual goals of the organization, and those who work for the organization itself. Examples in education would be teachers who work and sacrifice to teach children, vs. union representatives who work to protect any teacher including the most incompetent. The Iron law states that in all cases, the second type of person will always gain control of the organization, and will always write the rules under which the organization functions.
Robert Conquest's Third Law of Politics
The simplest way to explain the behavior of any bureaucratic organization is to assume that it is controlled by a cabal of its enemies.
Some prior posts on the topic in Thingfinder:
The Iron Law of Oligarchy spawns the Iron Law of Institutions
Posts
No comments:
Post a Comment