A canary in an epistemological coalmine

This is a rather tangled set of subjects but there are some interesting apsects. From Kardashian Index by Judith Curry.

Curry is commenting on an article by Neil Hall, The Kardashian index: a measure of discrepant social media profile for scientists.

Abstract

In the era of social media there are now many different ways that a scientist can build their public profile; the publication of high-quality scientific papers being just one. While social media is a valuable tool for outreach and the sharing of ideas, there is a danger that this form of communication is gaining too high a value and that we are losing sight of key metrics of scientific value, such as citation indices. To help quantify this, I propose the ‘Kardashian Index’, a measure of discrepancy between a scientist’s social media profile and publication record based on the direct comparison of numbers of citations and Twitter followers.

You could take Hall at face value. Does modern technology and social media potentially change things, including academic reputations? Sure. Are those changes potentially damaging as well as potentially positive? Again, Sure.

This is a case where close textual reading is beneficial to unearth that which has been hidden.

For example, what does it really mean to say that "there is a danger that this form of communication [social media] is gaining too high a value"? Does that mean that social media, as a tool can as easily distribute bad information as good? Surely not because that is true for all other media. And how would you know that it is "gaining too high a value"? What is the value you are measuring and where is the dividing line between enough and too much. The whole sentence carries very little information - all noise and no signal. All we can possibly infer is that Hall doesn't think it is a particularly wise idea for scientists to communicate via social media. That is not an unreasonable argument to make but you do have to make it directly rather than indirectly.

And is it true that we are "losing sight of key metrics of scientific value, such as citation indices"? Again, notice the unstated premises. 1) the existing citation indices are useful, and 2) that the existing citation indices are being used less. Are either of these premises correct?

Then we enter grade school argumentation. If you want to demean someone, you associate them with someone or something that is reviled. In common discourse there is always Godwin's Law - an argument is lost by the person who first compares their opponent to Hitler or the Nazi's. Technically:

As an online discussion grows longer, the probability of a comparison involving Nazis or Hitler approaches 1

The conclusion is that when a person advancing an argument reaches a point where they make comparisons to Hitler or Nazis, then it means that their forgoing arguments are insufficient and unavailing.

In this case, Hall hasn't quite reached a Godwin's Law level of anemic argumentation but he certainly approaches it. I assume that in academic circles, being compared to Kim Kardashian might have a similar effect as comparing someone in the general public to Hitler might have.

Here's what I find interesting. I think Hall is actually exploring an interesting and worthwhile topic. What is the relationship between academic standing (citations) and academic accessibility (social media usage)? These are independent issues that are not necessarily in conflict with one another. Is there a relationship? If so, what is the nature of the relationship?

But Hall comes across as an old guild member trying to protect ancient privileges of the guild rather than exploring an issue with an open mind. Is the old citation system a good measure of scientific impact. Like almost everything, there is more than one way to measure something, and there are always multiple ways to game the system. It depends on what your goal is.

If you want to measure impact within a particular scientific community, then citation isn't bad. But what if you want to measure cross-fertilization. A string theorist in physics might be highly cited among his/her peers but in no other field whereas an experimental physicist might easily be frequently cited in both physics and biology (and chemistry). Which is more influential? Different questions, different measures.

Hall's discussion rather undermines these interesting questions.

In an age dominated by the cult of celebrity we, as scientists, need to protect ourselves from mindlessly lauding shallow popularity and take an informed and critical view of the value we place on the opinion of our peers. Social media makes it very easy for people to build a seemingly impressive persona by essentially ‘shouting louder’ than others. Having an opinion on something does not make one an expert. But on Twitter, for example, the ‘top tweet’ on any given subject will not necessarily come from an expert, it will come from the most followed person. If Kim Kardashian commented on the value of the ENCODE project, her tweet would get more retweets and favorites than the rest of the scientific community combined. Experts on the Syrian conflict will tell you how frustrating that can be.

I propose that all scientists calculate their own K-index on an annual basis and include it in their Twitter profile. Not only does this help others decide how much weight they should give to someone’s 140 character wisdom, it can also be an incentive - if your K-index gets above 5, then it’s time to get off Twitter and write those papers.

There is a rank stench of status envy and privilege assertion. It comes across as "Just leave it to the experts, don't bother your pretty head about these complicated matters." And it is not as if we don't know and haven't extensively documented the epistemological dangers and shortfalls of relying either on Science (see the current furor regarding the large percentage of science papers withdrawn or not replicated) or on experts (see Nate Silver in The Signal and the Noise for a popular summary).

Curry quotes a contrasting view of the role communication (and social media) plays in the field of science from neuroscientist Micha Allen.

While a (sorta) funny joke, it is this point that is done the most disservice by Neil’s article. We (the Kardashians) are democratizing science. We are filtering the literally unending deluge of papers to try and find the most outrageous, the most interesting, and the most forgotten, so that they can see the light of day beyond wherever they were published and forgotten. We seek these papers to generate discussion and to garner attention where it is needed most. We are the academy’s newest, first line of defense, contextualizing results when the media runs wild with them. We tweet often because there is a lot to tweet, and we gain followers because the things we tweet are interesting. And we do all of this without the comfort of a lofty CV or high impact track record, with little concert assurance that it will even benefit us, all while still trying to produce the standard signs of success. And it may not seem like it now – but in time it will be clear that what we do is just as much a part of the scientific process as those lofty Nature papers. Of course – we are only fallible human beings, trying to find and create utility within a new frontier. We may not be the filter science deserves – but we are the one it needs. Wear your Kardshian index with pride.

Among Curry's commentary, she notes:

Here’s how I do the calculus for my own intellectual activities. As per google scholar, I have a total of 12,000 citations of my publications (since my first publication in 1983). Climate Etc. gets on average about 12,000 ‘hits’ per day, and 300-400 comments. I can spend my time blogging, discussing topics on which there is significant public interest, or I can write an academic paper, pay $1500 to get it published (hopefully in a high impact journal), so that 300 or so people can read it behind paywall. Since I am a senior tenured faculty member, I have the luxury of choosing to spend a significant amount of my time on social media outreach and engagement, which is growing my impact as a scholar in ways that I think matter.

All this is merely a morsel of the transformations wrought by the internet and which we are still coming to terms with.

Back in the early 1800s a number of European countries achieved mass/universal literacy and numeracy which had many consequences in terms of industrialization, politics, economics, voting rights, etc. Literacy and numeracy were the grease that eased a lot of historic transitions.

Not dissimilarly, I think the grease of constant easy access to information both broad and deep will likewise have similar effect today. What it means and where it will lead us is hard to discern. Hall is a little bit of a canary in the coalmine. His distress, status envy, and fear of change are likely portents of things to come.