Measurement of journalistic performance

Hmm. That's interesting. I came across the Society of Professional Journalists which has been around since 1909. The fact of its existence is not what is so interesting, it is their code of ethics.

I am a management consultant and therefore the capacity to measure and monitor is a central function of any process. If you can't measure it, you can't improve it.

There is nothing especially controversial about their code of ethics which is organized around four key principles: 1) Seek truth and report it, 2) Minimize harm, 3) Act independently, and 4) Be accountable.

Great principles. Under each, they have an elaboration of what each means. It is these elements which I find interesting because they create the opportunity for assessing performance. How well does a newspaper or journalist perform on each of these desirable attributes? It raises interesting questions. In a well run enterprise, your performance measures drive performance, behavior, incentives, etc. Do media companies use this Code of Ethics as part of their performance reviews for journalists or are these simply statements of virtue with no real world impact?

What I did was to turn the code of ethics into a checklist. I kept the performance assessment simple (Low, Medium, High). I then did an eyeball check of half a dozen New York Times news articles against these stated standards. The results were revealing. The biggest issue (on too small a sample) was sources. The more striking the news report, the fewer the named sources. In three of the articles there were no named sources. Balance was another major issue - reporting from one side or viewpoint only was the norm. Failing to distinguish reporting from advocacy was also common. On and on. The Low to High continuum was flawed. I needed a bottom category when the ethic was not adhered to at all.

Very revealing exercise. It would be interesting to run this exercise on a few hundred randomly selected articles across multiple news enterprises and with a more granular and defined performance scale. It would manifest Michael Crichton's observation about the Gell-Mann Amnesia effect.

Briefly stated, the Gell-Mann Amnesia effect is as follows. You open the newspaper to an article on some subject you know well. In Murray's case, physics. In mine, show business. You read the article and see the journalist has absolutely no understanding of either the facts or the issues. Often, the article is so wrong it actually presents the story backward—reversing cause and effect. I call these the "wet streets cause rain" stories. Paper's full of them.
In any case, you read with exasperation or amusement the multiple errors in a story, and then turn the page to national or international affairs, and read as if the rest of the newspaper was somehow more accurate about Palestine than the baloney you just read. You turn the page, and forget what you know.

Here are the four sheets. Click to enlarge.