AI with prejudices

Racist AI

In 2016, Microsoft presented Tay, a Twitter bot, an experiment in "conversational understanding. 'The more you chat with Tay,' Microsoft said, 'the smarter he gets, learning to engage people through casual and playful conversation.' Unfortunately, the conversations didn't stay playful for long. Pretty soon after Tay launched, people started tweeting the bot with all sorts of misogynistic and racist comments. In less than 24 hours, Tay, fueled by extremist Twitter talk, turned into a racist bully. Where did things go wrong? Clearly, AI models do not always produce the hoped-for results.

Based on the training set and human feedback, the algorithm increased certain weights and decreased others. When the development team finds that the control data produces the desired result, the model can be deployed in practice. How and why the AI model reaches a conclusion is no longer important at that point. The predictions match the desired results.

You cannot argue with AI models and argue why they have come to a particular conclusion.

But that's exactly where the problem lies. You cannot discuss with AI models and argue why they have come to a certain conclusion. The reason why seems to be a closed book, a so-called "black box.

IBM tried to promote its Watson supercomputer as an important cancer detection tool, but (human) oncologists simply did not trust the system. Two oncologists can argue, but you can't do that with an AI model. An AI cannot argue why it made a particular decision.

From input to output

In the images below, you get to see once again how an AI algorithm works: 

1. The input variable(s). 
2. The weight: if a particular variable is more important than another, reinforce the weight. This allows the algorithm to "learn." After all, if the output is wrong, the weights can be adjusted. 
3. The "bias" (you'll learn more about this below). 
4. The output.

Explainable AI 

Only by keeping AI algorithms and models explainable can AI development be controlled.

More and more, there is a growing demand that AI algorithms be explainable. It must be clear how and why a model arrives at a particular decision. AI critics and people who believe in a dystopian future where AI takes over the world are also advocating Explainable AI (XAI). Only by keeping AI algorithms and models explicable can the development of AI be controlled.

Suppose a credit card company wants to know if someone is creditworthy. A developer can deploy an AI algorithm to determine whether this is the case for any given user. But creditworthiness is a subjective and fuzzy concept. A person is less creditworthy if the risk of defaulting on payments is higher (for example, low salary or high risk of unemployment). However, the lender in question can earn more from it than from traditional loans.

The company will initially want to make as much profit as possible by granting as many loans as possible to be repaid at maximum interest rates. 'Creditworthiness' is thus reformulated on the basis of the highest possible profit margin. If the algorithm discovers that granting loans to less creditworthy people is an effective way to maximize profits, it will eventually engage in predatory behavior, even if that was not the company's intention.

Bias

Bias, "bias" in English, can occur when the data you collect is not fully representative of the real situation or reflects existing biases. What if you develop a piece of AI software for recognizing faces, but only feed faces of white people to the system as training data?

Amazon developed a job application tool that very clearly favored male candidates because it learned from historical data from the company. What attributes or characteristics you will use in the training data help determine the end result. Take the Amazon job application tool as an example: you can also use the number of years of experience or education as an attribute as input. In the case of creditworthiness, age and income play a determining role, not so much earnings to be gained. These biases are proving more difficult to resolve than thought.

What are the causes of AI bias?

1 Unknown unknowns

Often developers do not determine the effects of the chosen attributes until after the fact. In Amazon's case, the developers reprogrammed the tool by excluding all gender-related words from their data. But even then, the modified system picked up verbs that correlated with men rather than women.

2 Training and control

Often developers split their data in advance into a training set and a control set. This means that the control data contains the same biases as the training data.

3 Lack of social context

Computer scientists or developers mainly want to develop systems that can be used in multiple situations. Precisely because they disregard the social context, you get biases. You cannot develop a job application tool for Amazon and use it likewise for recruiting and selecting construction workers.

4 How do you measure honesty?

When is there fairness and prejudice? Is Black Pete racist? Biases occur everywhere and all the time, but everyone defines them differently. They also occur in AI data. The difference, however, is that for AI they are mathematically defined.

How can you prevent/resolve such biases?

Does this mean that you have to make groups appear equal in all situations? Should women and men, people of white or colored skin, and so on, be equally represented in all situations? Should women score equally high or often in job applications, even if the groups do not meet the other necessary requirements?

AI researchers are doing their best to address the problem of "AI bias. They are trying to develop algorithms that help detect and reduce hidden biases within training data or reduce model biases regardless of data quality. Simple cannot be solved. It is a continuous process and we must always be aware when developing AI tools that the results or predictions are not just the result of a mathematical algorithm. As in all other aspects of society, predictive analysis is never completely objective.