The mismeasurement of education

Among the many crises of education is a crisis of measurement - we don't know what to measure, we don't measure it very well, and we don't quite know what the data we do have actually tells us.

From HS Graduation Rates Go Up Even as Students and Teachers Fail to Show Up by Max Diamond.

Phelps Architecture, Construction and Engineering High School in Washington, D.C., seemed to be doing great: A public school that enrolls only economically disadvantaged students, its graduation rate hit a recent high of 94.5 percent during the 2016-17 school year.

Yet in that same year, three-quarters of the students at Phelps were absent more than 10 percent of the time.

Phelps reflects a national trend in which high schools across the country have both high absenteeism and high graduation rates. A recent national study by the U.S. Department of Education showed that about one in seven students missed 15 days or more during the 2013-14 school year – the year before the national high school graduation rate hit an all-time high of 84 percent.

Students aren’t the only ones not showing up – absenteeism is also common among teachers. The Thomas B. Fordham Institute, an education think tank, found that in 2013-2014, at least one-fifth of traditional public-school teachers missed more than 10 days in 32 of the 35 states studied. According to federal data, in 2015, more than 41 percent of Rhode Island’s teachers were absent more than 10 days of the year. That was an increase from under 40 percent in 2013, but Rhode Island’s graduation rate nevertheless has hit an all-time high.

"It’s really easy to graduate more kids,” said David Griffith, a policy associate at the Fordham Institute. “You just graduate them.”

A good education is a high-stakes enterprise. You learn skills, accumulate knowledge, flesh out your capabilities, acquire social norms, etc.

And yet. . . We really don't have particularly good or sophisticated mechanisms to measure how well children are learning. We are reluctant to address social norms. We struggle distinguishing a good teacher from mediocre from bad. We have no idea about how to measure the gap between potential and achievement. Of the measures we do have, most are easy to game (grades, graduation, etc.). The one measure which is reasonably solid, IQ, is widely reviled in some quarters.

Earlier this year, we discovered that the rapidly improving graduation rates in Washington, D.C. were achieved by good old-fashioned chicanery. A few years ago, we discovered that the much heralded improvement in Atlanta Public School standardized tests was the result of good old-fashioned cheating - by teachers and administrators. And these were just the flagship scandals - the few crystalized instances of what was known to be happening extensively in other school systems.

So what Diamond is reporting is not especially surprising (I have a number of posts related to education and graduation). But it is a new wrinkle. While measuring how much student learn can be challenging, measuring whether they are there or not is relatively straight-forward. It is a straight-forward easily determinable thing to measure, though it can still easily be manipulated.

None-the-less, Diamond is pointing out that attendance measures are serving as a proxy red flag. How can graduation scores be improving while at the same time chronic absenteeism is also increasing. Hypothetically you can create a number of scenarios where that could be true, it is just unlikely to be true.

We keep trying to centralize and standardize learning across regions and cultures and values and circumstances that defy standardization. While I value education highly, and am enamored with the romantic idea of a shared community education system, the reality of public school seems to support ever more radical reforms. Left to themselves, the enterprises of public education always seem to work agendas for stakeholders other than those of interest to parents and their children.

It is a system that seems destined to radicalize citizens.