Saturday, December 12, 2015

We don't know what we are talking about for a long time before we are in a position to make good decisions

Real Clear Policy juxtaposes two essays, one in favor of the proposition that diversity leads to enhanced educational outcomes and one arguing against it. Diversity actually makes us smarter by Amy X. Wang is the essay that argues that there are measurable benefits. The Educational Benefits of Diversity Are Dubious by Jason Richwine is the essay that there is little evidence that there are measurable benefits.

I'll preface the following comments that I believe all systems - economic, political, social, educational, etc. - need some degree of variance (need diversity) in order to evolve under changing exogenous circumstances. I believe diversity to be strategically critical, in the right proportions, in the right circumstances. However, that doesn't mean that diversity is tactically beneficial. Indeed, in most instances, it is tactically inefficient while still being strategically necessary. Most arguments about social diversity never even broach the distinction between strategic value and tactical value and how they can sometimes work in opposite directions.

These two essays are no different than others. In reality, based on both essays' argument, there is no reliable evidence one way or another. We are all arguing from ideological belief and anecdote.

Wang leads with the assertion that
With those kinds of conflicts as backdrop, it’s worth noting that a large body of research—conducted across dozens of years, countries, and situational settings—maintains that racial and ethnic diversity is critically important to our communities, our social institutions, and even our own brains.
That's interesting. I'd like to see that large body of research. From my autodidactic perspective, there is only a small body of research, the results are often contradictory and the effect sizes tend to be very small. So where is the large body of research that I have been missing all these years.

Instead of marshalling that evidence, Wang leads with anecdotes and isolated small studies. She has lost the battle with her opening shots. She doesn't actually have the evidence she claims, thus discrediting her argument further, i.e. not just lacking in evidence but lacking in good-faith.

Wang's lead example doubles down on the self-evident weakness.
Take these examples. In 2003, researchers found banks with racially diverse employees yielded better financial performance than those that didn’t.
Usually in an argument, you lead with your best shot. If this is the best, there's no argument to be had because the finding does not address causal direction. Does a prosperous bank go out of its way to hire a more diverse workforce (lots of evidence for that) or does a diverse workforce make the bank more successful (virtually no evidence, anecdotal or otherwise, to support that causal direction). Prosperous institutions are notoriously faddish. They can afford to be. Diversity, recycling, green initiatives, employee engagement, community involvement, maternity leave policies, employee fitness programs, etc. - these are all brand signalling strategies that prosperous companies undertake to cultivate an image that they believe will be beneficial to them in the marketplace. But the prosperity came before the initiative. You have to show causation, not simply correlation and if Wang doesn't understand the difference between correlation and causation and the importance of causal direction, then it probably doesn't warrant further reading. She has no credible argument.

Further, Wang never defines diversity and many of her anecdotal examples are related to experiential or cognitive diversity and not racial diversity. Much of her anecdotal evidence is qualified by weasel words such as "suggests" and many of the experiments are lab based. She mentions neither the size of the experimental groups or, except in one instance, the effect sizes.

So the argument for the proposition that there are measurable benefits to racial diversity in social systems is sunk by the typical issues in social sciences research: too small sample sizes, not reported effect sizes, poor experimental protocols, failure to establish causal direction, failure to define critical variables, failure to control critical variables.

What about the case for the propositions that there are no measurable benefits to racial diversity in social systems.

Richwine is actually arguing, per the headline, that the The Educational Benefits of Diversity Are Dubious.

Here the error is on the part of the Real Clear Policy editors. These two articles are treating a similar issue but they are not addressing the same argument. Wang is arguing that we know that racial diversity in social systems is beneficial and Richwine is arguing that we don't know whether racial diversity in social systems is beneficial.

Richwyne actually has the better argument. He points out
The premise of the affirmative-action case currently before the Supreme Court is that ethnic diversity improves higher education. But how do we know the premise is true? It is surprisingly difficult to find solid evidence in the case documents. The original UT–Austin brief refers me to the Joint Appendix. The Joint Appendix in turn cites the “Affidavit of N. Bruce Walker,” admissions director for UT–Austin. In that affidavit, Mr. Walker says that diversity improves race relations “in the University’s judgment.” He also assures us that, “I have read studies that tout other benefits.” Those studies are not listed.

The amicus briefs in the case are more forthcoming, but the barrage of studies they cite often have tiny effect sizes and narrowly focused results, making them more suggestive than convincing.
Good point.

Richwine doesn't make it explicit, but there is another problem in the pro argument. He mentions
A good example of narrow focus is yesterday’s New York Times op-ed by Sheen Levine and David Stark. The authors describe how the distrust and friction generated by ethnic diversity can be beneficial in avoiding the groupthink that creates stock-market bubbles. Interesting point. But then they go way beyond their empirical findings to conclude: “Ethnic diversity is like fresh air: It benefits everybody who experiences it.”

In fact, Levine and Stark’s research demonstrates how theoretically ambiguous the impact of diversity can be. Interpersonal friction may have some benefits, but it also carries obvious costs. It is especially difficult to reconcile the diversity benefits that Levine and Stark describe – which depend on people distrusting each other – with the supposed improvements in cross-racial understanding that come from diversity. The literature contains evidence of diversity producing undesirable conflict, and the uproar over Halloween costumes at Yale could probably be a new case study.
He's right as far as he goes, this is evidentiary overreach. But there is something more fundamental amiss here. The pro advocates, as almost all advocates do, present only the benefit side of the equation and not the cost side. We are interested in how big the net benefit might be after accounting for both costs and benefits.

Just as all systems strategically require some level of variance in order to evolve, all system changes come with costs and benefits.

Sometimes the costs are negligible and the benefits large and sometimes the reverse. Regardless of the relative size of the costs and of the benefits, you have to take into account both to get to a net assessment whether the proposed change is warranted.

In this argument, those arguing that there are inescapable benefits and no material costs have virtually no case to make. This is simply a political game over power to coerce at this point in time.

The question remains legitimate but we are a long way from answering it. We cannot make informed decisions at this point.

No comments:

Post a Comment