Illustrating how AI discredits journalists and forces clarity of human thinking

In the past couple of days I have commented at least a couple of times on the obsessively skewed news reporting which we are currently witnessing. See: Deranged Hysteria and Millions of people become simultaneously impressed with one delusion, and run after it, and Moral arbitrage.

Moral arbitrage looks at extreme claims and emotional communication from an economics of politics perspective. Dramatic, but false social justice claims are rewarded in some circles with increased prestige, or at least admission into in-group status among the coercively altruistic.

And this afternoon along comes an excellent example of just this bad reporting in the instance of AI Programs Are Learning to Exclude Some African-American Voices by Will Knight. Knight casts this in social justice terms and makes two separate claims which get conflated. The first claim is that AI is specifically discriminating against African-Americans because of their accents. The second claim is that AI systems are prone to discriminate against African-Americans for reasons other than accent. A mildly critical reading indicates there is no evidence offered for either claim. It is not hard to see the problems.

All too often people make snap judgments based on how you speak. Some AI systems are also learning to be prejudiced against some dialects. And as language-based AI systems become ever more common, some minorities may automatically be discriminated against by machines, warn researchers studying the issue.

Anyone with a strong or unusual accent may know what it’s like to have trouble being understood by Siri or Alexa. This is because voice-recognition systems use natural-language technology to parse the contents of speech, and it often relies on algorithms that have been trained with example data. If there aren’t enough examples of a particular accent or vernacular, then these systems may simply fail to understand you (see “AI’s Language Problem”).

AI systems are not learning to be prejudiced against African-American dialects as indicated in the headline. AI systems are having difficulty with all variant accents (not dialects and not African-American accents selectively). If you are a southerner, Jamaican, a Bostonian, or from the Bronx, rural, upper midwestern, perhaps even Canadian, Australian, Scottish, Irish, you will initially have trouble because your accent is variant from the received norm, and natural language processing systems will have difficulty processing your articulations. The stronger and more variant the accent, the longer it takes the system to learn. The beauty of AI is that it does learn. The more input, including the more variant input, the faster it learns and the more sophisticated it becomes.

When Siri first came out, my wife, with her markedly South Carolinian accent, nearly got into a fistfight with Siri owing to Siri's rugged determination to insistently misinterpret her. Here is the Scottish duo from Burnistoun illustrating the challenges of voice recognition software with the Scottish accent (click link for YouTube display).



The headline to Knight's article is an intentionally misleading click-bait journalistic stretch to drive readership. The startling thing is that the second and third sentences of the article explain exactly why this is not an African-American issue. It is a general issue. But you have to have a modicum of knowledge about AI to recognize how pernicious is the misreporting.

"Some AI systems are also learning to be prejudiced against some dialects." NLP systems are trained on some standard set of spoken English norms, whether British Standard English with Received Pronunciation or some Mid Atlantic/Midwestern version in the US. There are so many variant accents that it is impossible to pre-train NLP on all possible variations. And of course there are all sorts of stop-points such as how to address pidgin-English and other creole versions, how to recognize variant idioms, etc..

The initial base-training is released and, with AI, the system is gradually able to acquire a greater and greater comprehension of the variant pronunciations and accents. This has nothing to do with African-Americans as Knight and the headline writer ought to know. It is a bog standard issue of natural language processing and AI dealing with variances from the norm.

How about the second claim that AI systems are prone to discriminate against African-Americans for reasons other than accent? Again, tosh. Here is the claim:

The issue of unfairness emerging from the use of AI algorithms is gaining attention in some quarters as these algorithms are used more widely. One controversial example of possible bias is a proprietary algorithm called Compass, which is used to decide whether prison inmates should be granted parole. The workings of the algorithm are unknown, but research suggests it is biased against black inmates.

In this instance, the problem is the loose or undefined usage of "bias". In the vernacular we tend to mean that it is an unwarranted assumption against someone whereas in statistics it means the skew in data one way or another. Knight is conflating the two terms which obfuscates what is going on. There is no way of knowing whether this conflation is through ignorance or deliberate deception.

Lee Jussim is one of the leading researchers in the field of Stereotype Accuracy. The challenge is that for any socioeconomic measure there are going to be accurate statistical skews in the data patterns and which also correlate with race, gender, orientation, religion, ethnicity, national origin, age, health, etc.

Some of this we recognize and accept. No one has a problem with accepting that younger drivers have more accidents than older drivers or that male drivers have more accidents than female drivers, or, in a perfect storm of bias, that younger male drivers have more driving accidents than any other demographic cohort. It is a bias in the data (in statistical terms) and it is an accurate bias.

The challenge when you mix statistics and people is maintaining definitional clarity. It is fully accurate, that on average, young males are the worst drivers in terms of accidents. It is also true that that tells you nothing about any individual young male driver. There are certainly some sterling young male drivers with impeccable driving records. Just fewer than the average. Humans have a terrible time keeping the real distinction between what is true for the average and what might be true for an individual. Yet, to be fair, that is what we always should do.

Part of the reason that it is hard to maintain the distinction between the group average and the individual is that we often, and usually for good reasons, ignore the distinction when the risk is perceived as too high.

Say you contract drivers for the local school district bus system. You have two candidates with equally spotless driving records. One is an 18-year old male and the other is a 35-year old mother. You know from group actuarial statistics that the 18-year old male is, on average, much more likely to have an accident than the 35-year old mom. By law you cannot discriminate. By principle, they are equal candidates based on past performance. But on average one choice is more likely to lead to an accident than the other choice. Do you take into account the actuarial reality (which is based on group averages) or do you focus only on the law and the record to date? The law doesn't actually help much. On the one hand, it says that you cannot discriminate based on sex or age. On the other hand, civil law exposes you to law suits if there is an accident and you "negligently" went with the young man.

All of us, routinely, in innumerable decisions large and small, impinge on the fairness of a process, and break the barrier between the individual and the average in order to take into account probabilistic future outcomes which have nothing to do with the individual.

It is complicated when humans do it. But at least with a human there is a chance that an individual approaches such a decision with a 360 degree perspective on the facts, the law, the probabilities, and the ethics.

But even with humans, the claim of systemic discrimination turns out to be a function of real world variance in the data. There is a common claim that African-American borrowers are discriminated against in bank loans because they are denied more frequently. However, those researchers who have looked into this generally find that such accusations of discrimination don't take into account the full picture. Apples are not being compared to apples. You have to consider income, credit history, other capital sources available as security, stability of past income, and various other factors and hold them all equal when doing comparisons between black and white.

It is very similar to the frequently cited gender wage gap which makes a claim of discrimination based on the total income per woman being less than the total income per man without taking into account career field preference, work related danger premiums, full-time versus part-time, fixed hours versus variable hours, flexibility for overtime or travel commitments, duration of work record, past objective work accomplishments etc. Here and in Europe for thirty years, whenever the research takes into account all the variables, there is no discrimination.

The problem for AI is that it will inevitably find variance in patterns on any known factor such as race, religion, sex, orientation, age, etc. It will point out differences we prefer to not acknowledge or otherwise obfuscate. The fault is not in the AI but in our own inconsistency in what we prefer the data to tell us versus what the data actually tells us.

And again, this is not a racial issue unique to African-Americans. There will be variances of one sort or another for any socioeconomic variable that correlates with race, gender, orientation, religion, ethnicity, national origin, age, and health. AI is forcing us to confront issues we would sweep under the rug. If AI sees from the data that young male drivers have a pattern of more accidents than others, is that a good thing or is it a problem? Neither. It is reality and the degree to which we wish to take account of that reality is a human decision, not an AI problem.

Knight could have written an interesting article about how AI is revealing patterns of behavior correlated with socioeconomic variables and the challenges that that poses for clear human decision-making. Instead he wrote a click-bait article advancing cognitive pollution based on popular narratives of race and discrimination.

The problem is with humans not with AI.


UPDATE: AI 'Bias' Doesn't Mean What Journalists Say It Means by Chris Stucchio and Lisa Mahapatra makes much the same points as above but more grounded in the data.