You have to get the diagnosis right to get to a good remedy

An interesting example of talking at cross-purposes as well as of epistemic closure.

Ben Casselman is a reporter at the Fivethirtyeight site which is dedicated to bringing empirical evidence and quantitative analysis to discussions. Casselman seems an OK reporter from my reading experience but representative of the media in that he is carries certain assumptions that might not be widespread outside media/academia and subscribers to the various postmodernist, critical race theory, critical theory, third wave feminist, post-colonial theory ideologues. I don't think he likely subscribes to many of the extreme positions of these ideologies but has likely unquestioningly absorbed some of their assumptions and precepts.

He gets called out and tangled up in a somewhat innocuous article, Mizzou’s Racial Gap Is Typical On College Campuses by Anna Maria Barry-Jester and Ben Casselman. Their opening paragraph:

Thousands of college students across the country on Thursday joined protests demanding that their schools do more to address racism and discrimination on campus. Students at many campuses are calling for an increase in the diversity of faculty and more resources to help minority students succeed. The data suggests that they have a point: Colleges across the country are far less diverse than the communities they serve.

Anna Maria Barry-Jester and Ben Casselman support their conclusion with a useful visualization of their data analysis.

The argument here is that minority students claim they are underrepresented and that they are correct that they are underrepresented. There is an implied corollary; that underrepresentation creates some measurable problem for the minority students. There is the further argument, that the underrepresentation must be the result of either conscious or systemic discrimination.

In the comments, commenters are pointing out several flaws in the article and the accompanying analysis. The general commenter perspective seems to be that by analytically attesting to the accuracy of underrepresentation, Casselman is also endorsing the wider student argument, i.e. that underrepresentation is both a problem in itself and that it is the consequence of racial discrimination.

I don't think that is a correct reading of Casselman's argument but it is easy to see how he was not sufficiently clear about divorcing the factual issue of underrepresentation from the broader argument being made by the protesting students.

But Casselman does open himself to more substantive criticism by his flawed analytical methodology which breaks several standard scientific method protocols. The main areas of weakness are data manipulation, failure to define relevant populations, failure to compare like-to-like, and ignoring contextual conditions that invalidate his argument.

We looked at 2013 data from the U.S. Department of Education for four-year universities with heavy research activity, a proxy for the selective schools that are Mizzou’s peers. We excluded historically black universities and colleges (HBCUs), which have much higher black enrollment and skew the overall figures.

This is substantially an issue of framing and Casselman's chosen framing is representative of general biases in the media.

This paragraph contains the first big flaw. In fact, two flaws. The first is the restriction of the data set to selective four-year universities with heavy research activity. The problem isn't the restriction per se, but the restriction without justification. Why are you looking just at these type of universities which are a small portion of all colleges and universities and by definition are distinctive in nature and therefore likely unrepresentative. You can make an argument, but it has to be made explicitly, not left to the imagination of the reader.

The second major flaw is that "We excluded historically black universities and colleges." To see why this is an issue consider a hypothetical. Assume African-Americans attend university at the same rate as everyone else, that they perform equally well, and that there is no discrimination. Also grant that there are historically black universities and colleges which attract some portion of the African-American student population. By logic and mathematics, all other non-historically black universities and colleges will have an underrepresentation of African-American students to the same degree that African American students choose to attend historically black universities and colleges. If 13% of African-American students choose to attend historically black universities and colleges, then all other universities will have a 13% underrepresentation of African Americans. Not because of any action on the part of the universities but because of choices made by African American students.

Casselman, by excluding historically black universities and colleges, has instantly begun validating the protester's argument that there is underrepresentation but without acknowledging that some portion of that underrepresentation is due to the choices of the students and has nothing to do with discrimination. For a rational empiricist and anyone more right of center, this makes Casselman look like he is sympathetic with the student's unsupported argument. It is unacknowledged biases of this sort, I think, which generates the overwhelming public perception that the media is overwhelmingly biased and cannot be trusted.

There are further issues in the framing. Casselman not only focuses exclusively on selective four-year universities with heavy research activity, but he fails to provide the overall picture. African Americans are about 13% of the population. According to the government, 15% of college students are African American. So they are not, in fact, underrepresented overall, but only in some sectors of the higher education establishment. With this context, the picture becomes quite a bit murkier.

It becomes murkier still when we turn to the final major analytical flaw in Casselman's analysis and that is comparing apples and oranges.

What are the two base requirements of attending a competitive college? 1) You have graduated from high-school and 2) You are within a couple standard deviations of the mean IQ/SAT/ACT for that particular institution.

In a multicultural society, different groups value different goals and behaviors differently, including the valuation of education attainment. To the extent that culture correlates with ethnicity (not always though), you would then expect there to be a variation among ethnic groups in college matriculation and graduation. There is an eleven percentage point gap in the high school graduation rates between African Americans and the national average.

It is not quite as simple as that, but for simplicity's sake, let's make the leap that a 11% deficit in graduation rates corresponds to a 11% reduction in college attendance. So now we have two sources of underrepresentation having nothing to do with university actions - 13% owing to the number of African American students who choose to attend historically black universities and colleges and 11% owing to lower high school graduation rates.

The next issue not addressed by Casselman in getting to a point of comparing like-to-like is that there is the very material variation in SAT scores between ethnic groups in the US. Not so much an issue at the overall average, but given the gaussian distribution of scores, the mean differences have an exponential impact at the most competitive universities where most the students are in the far right tail of the distribution where the differences in group means are most consequential.

When you take into account the reduced number of candidates students because of differences in mean SAT scores, AND the differences owing to student choices regarding attending historically black universities and colleges, AND the differences arising from lower high school graduation rates, I suspect that the putative racial gap disappears substantially or completely. Indeed, with residual affirmative action programs still in place, it is not improbable that African American students are overrepresented in selective four-year universities with heavy research activity.

One final complicating contextual piece of information which Casselman omits but which is relevant to the larger protester argument. Particularly among the most competitive universities, 25-40% of African American students enrolled are actually either African or are first generation. So the issue does not appear to be racial discrimination per se on the part of universities but rather to do with cultural orientation, behaviors and demonstrated abilities.

Casselman ends up writing an article that purports to support at least one leg of the student argument, that African Americans are underrepresented in competitive universities. He (and the students) are correct that they are underrepresented in terms of the general population. He gets into two different pots of trouble. One pot is that he fails to make clear that his attempt at validation of a single fact in the students' argument does not reflect an endorsement of the entirety of their argument, particularly the accusation that the underrepresentation is a product of racism on the part of the universities.

The second pot of trouble is that his purported validation of the the fact of underrepresentation is too glib and subject to rigorous fact checking. He omits important data (HBCUs). He fails to define the comparison pools (comparing underrepresentation against the general population instead of the relevant population which are high school graduates who are college ready at a certain achievement level (Ex. SAT scores)), and finally, he ignores counter evidence such as the success of foreign born black students in the most competitive US universities.

The reason that this failure on the part of Casselman is important is two-fold. One is that such failures of analysis fuel the general perception that mainstream media is not to be trusted to convey facts and that the media is left-biased.

The second reason for the importance of this failure is that it supports continued misdiagnosis of the problem. As long as we simply accept the argument that all the problems must be due to administrative action or inaction on the part of universities, and more particularly as long as we accept that such universities are racist, then we fail to tackle the real issues. As long as we support HBCUs, then there will always be an underrepresentation of African American students in universities. That is simply a trade-off choice that is unpleasant but has to be made. More substantively, if the underrepresentation is substantially due to low graduation rates and poor academic achievement, then no action at the university level is going to likely make a material beneficial difference. To solve the problem, you have to diagnose it correctly. By failing to do so, and by appearing to support a misdiagnosis, Casselman, and journalists of his ilk, fail their readers and the citizens of the nation.