Maya's phone has a photo app that sorts pictures by face. It finds her kids in every photo, even blurry ones.
But it keeps mixing up her mother and her aunt. It cannot seem to find her father-in-law at all.
Her son thinks it is funny. Maya is not so sure. Why does the tool work well for some people in her family and not for others?
In week 1 you learned that machine learning finds patterns in very large amounts of training data. People usually supply that data.
AI4K12, the education project from week 1, adds a warning. Biases in the data used to train an AI system can lead to some people being less well served than others.
So a tool can work well for one person and poorly for the next. The difference may come from who was in the data.
This course uses the word bias in a plain way. Bias is when a tool works better for some people than others, or judges some people unfairly.
Maya's story is made up for this lesson. It shows the kind of question to ask, not a fact about any real app.
NIST's AI Risk Management Framework lists what makes an AI system trustworthy. Fairness is on the list, next to safety and privacy.
Notice the wording: "harmful bias managed." The framework does not promise a tool with no bias at all.
It asks that harmful bias be found and managed. That takes people who look for it.
| Statement | True or false? |
|---|---|
| A tool can work well for some people and poorly for others. | ? |
| NIST lists fairness among the traits of trustworthy AI. | ? |
| The NIST framework asks for harmful bias to be managed. | ? |
| If a tool works well for you, it works well for everyone. | ? |
Sources: AI4K12, Five Big Ideas in Artificial Intelligence (poster, version 2) · NIST AI 100-1, AI Risk Management Framework (2023)
Tomorrow: how bias gets into a tool, step by step.