Community forums with quality commenters can be a great place to surface new information

From The Long-Run Impacts of Same-Race Teachers by Seth Gershenson, Cassandra M. D. Hart, Joshua Hyman, Constance Lindsay, and Nicholas W. Papageorge. From the Abstract:

We examine the impact of having a same-race teacher on students' long-run educational attainment. Leveraging random student-teacher pairings in the Tennessee STAR class-size experiment, we find that black students randomly assigned to a black teacher in grades K-3 are 5 percentage points (7%) more likely to graduate from high school and 4 percentage points (13%) more likely to enroll in college than their peers in the same school who are not assigned a black teacher. We document similar patterns using quasi-experimental methods and statewide administrative data from North Carolina. To examine possible mechanisms, we provide a theoretical model that formalizes the notion of “role model effects” as distinct from teacher effectiveness. We envision role model effects as information provision: black teachers provide a crucial signal that leads black students to update their beliefs about the returns to effort and what educational outcomes are possible. Using testable implications generated by the theory, we provide suggestive evidence that role model effects help to explain why black teachers increase the educational attainment of black students.

Whether it is race or gender, I have seen many studies such as this over the years - black students do better with black teachers and male (or female) students do better with male (or female) teachers.

It certainly seems plausible; it easy to conjure reasons why it might be true. On the other hand, I strongly and instinctively don't want it to be true. I cannot but be concerned about findings which seem to support the return of race and gender segregation in education.

And I say this as a product of a single-sex high-school education which I suspect was superior to the alternatives.

I have never resolved this tension between what the data implies versus what I wish to be true.

One palliative thought has been that even if the studies are true, they don't necessarily require a return of segregation. Perhaps it just implies certain considerations in student class assignment within a still integrated school. Perhaps. But I don't take much solace in that prospect - it still feels like imposing the steel hand of identity regardless of individual needs.

The more reassuring thought has been that perhaps the studies are wrong. Certainly, the methodologies and protocols often seem quite questionable and the effect sizes relatively small. The fact that the studies seem consistently in one direction might be because of publication bias and/or the bias of educational academia. Academia is one of the bastions of postmodernist critical theory and its commitment to identity. It is not too surprising that they might be finding what they have already decided must be true.

So I am left in a quandary. I want to go with the evidence but the evidence as it exists is in a direction I hope is not true and which I have good reason to question.

The NBER paper is via Marginal Revolution. The nice thing about MR is that Cowen and Tabarrok's commenters are pretty high quality. There is a share of trollers and low quality comments, but in general, regardless of the topic, MR commenters drive toward greater understanding and clairty.

The commenting discussion on this NBER paper is pretty robust, raising all sorts of methodological, analytical and statistical issues casting doubt on the conclusions.

But then commenter Dave finds a pretty devastating critique already in circulation.

Dave
November 20, 2018 at 2:51 pm

Any thoughts on this critique of the study which seems to me to be relatively devastating? (Found via a Jesse Singal tweet):

First, the effect was limited almost entirely to enrollment in community college. Second, although black children with black teachers were, overall, 4 percent more likely to enroll in college, they were only 2.7 percent more likely to stay past their first year—and this result was only marginally significant. Third, they were only 0.5 percent more likely to earn a degree. The authors confirm this in the text—”We find a near-zero, statistically insignificant effect on degree receipt”—but also offer up some gobbledegook to suggest this is no big deal: “However, given the very low rates of degree completion among non-matched students (8.5%), we cannot rule out degree receipt effects on the order of 1 or 2 percentage points. Effects in that range would suggest the marginal matched student induced into college persisting to degree receipt at around the same rate as the inframarginal, non-matched student.”

If I’m reading that correctly, it means the effect on earning a degree is about zero no matter what.

Thank you, Dave. Follow the link to Kevin Drum's assessment.

Drum points out this graphic from the paper:

Click to enlarge.

5% more likely to graduate from high school is a reasonably good outcome, probably representing a real effect and one that is consequential in terms of life time earnings (HS graduates earning some 30-40% more than non-HS graduates).

However, 0.5% increase in the number completing college is a negligible effect-size and within the confidence margin.

And in terms of consequence, it is actually on the face of it, a net negative. Attending college has a time and dollar cost and the income earning benefit is almost completely dependent on graduation. Matriculating without graduating means you are out of the workforce for 1-3 years with the lost income that entails. In addition, if you take on any non-dischargeable loans to attend, then you are left with all the costs and virtually none of the benefits, a condition in which 85% of the newly attending student would find themselves (4-0.5/4).

The real prize in terms of improved income is a completed college degree and having race-matched teachers does not facilitate that big win. However, we are still left with that 5% increase in those who graduate high school, which would still be a real productivity win if it were true.

But we are left with serious questions - is that effect size large enough to be true? And even if true, is it sufficiently beneficial to outweigh the negative impact on the 3.5% who attend but do not graduate from college.