Loss of signal strength or loss of trust?

From The Decline & Fall of the US News Rankings by Brian L. Frye and CJ Ryan.  From the Abstract.

Have the U.S. News & World Report law school rankings become irrelevant? The ostensible purpose of the US News law school rankings is to give prospective law students convenient and reliable information about the relative quality of law schools and help them decide which law school to attend. Law schools care about the US News rankings because prospective law students care about the US News rankings. A ranking increase means more prestige and better credentialed students, while a ranking decrease means less prestige and students with worse credentials. Accordingly, law schools are jealous of their US News ranking.

Do prospective law students actually care about the US News rankings anymore? We compared changes in law school US News rankings to changes in prospective law student preferences the following year. Those variables should be strongly positively correlated. If a school’s US News ranking increases, prospective law students should prefer it more the following year, and if it decreases, they should prefer it less. But in fact, they were at best very weakly positively correlated, and often they are weakly negatively correlated. In other words, prospective law students appear to be largely indifferent to changes in a school’s US News ranking. This suggests that prospective law students are getting information about which law school to attend from someplace other than US News. And it also suggests that law schools can safely stop paying attention to the US News rankings, because their customers don’t care.

I don't think their conclusion follows from their evidence.  Sometimes, participation is required as a ticket to the table.  You might not need to have the best US News ranking, but perhaps you do need to have one.  Its a ticket to the table even if it is no guaranty of a win.

What would be especially interesting is if the researchers had extended their work back into the 1970s.  All they have actually done is demonstrate that there is little correlation between ranking and applications since 2014.  

What we really want to know is whether the rankings ever served as a useful signal of quality and desirability.  If they never served as a useful signal, then the fact that they did not in the past decade is not especially meaningful.  But, in their heyday in the 1970s or 1980s, if the rankings did serve as a useful signal, then the decline into irrelevancy now is an interesting fact.

It goes to the larger issue of declining trust in institutions and increased skepticism in signaling from sources whose financial well-being is tied to that signaling.  

The researchers assume that the rankings have become a weaker signal because everyone now has more and better access to information than in the past.

Prospective law students have a lot more of it, so they aren’t as reliant on the US News rankings, which gradually became less salient to them.

Maybe.  That would make sense if the ratings used to be strongly correlated with student applications.  But if there never was a signal in the first place, then increased information is irrelevant.  Further, the information access explanation might be irrelevant if the core issue is a loss of trust in the signal.

I am inclined to believe that perhaps there did used to be a stronger signal than there is now but I would like to know that there was in fact a strong correlation rather than merely assuming that there was one.  Further, if there is a real loss in signal strength, this research does not shed light on whether the loss is due to better access to other better signals or whether it is due to loss of confidence in the quality of the US News signal.