Teacher Mara spreads her Spanish students' practice sheets across the table.
"My students study late, and many of them work hard on their own," she says. "If SH-1 tells them a wrong verb ending, they will learn it wrong."
Comet thinks aloud. "But they could just use it less, right?"
"Only if they know it is weaker in Spanish," Wren says. "The packet says it works for every class."
Nova hovers over the packet cover and highlights the claim. "Would you like a hint?" she asks. "What happens when people trust a tool to work the same for everyone?"
Comet reads the cover again, slowly this time.
Biased training data can leave some people less well served than others.
A generative model can perform less well in languages other than English, or in some dialects.
People may wrongly trust it to work equally well for everyone.
That can leave the groups it serves worst with worse results than if no AI were used at all.
Bias is broader than having balanced numbers of people in the data.
AI systems can spread bias faster and wider than before.
NIST reports one example in words. Asked for pictures of CEOs, doctors, lawyers and judges, current image generators underrepresent women, racial minorities and people with disabilities.
The crew does not need to see any such pictures. The report shows that bias can appear in what a model makes, not only in its test scores.
| Statement | True or false? |
|---|---|
| Bias is only about having balanced numbers of people in the data. | ? |
| AI can spread bias faster and wider than before. | ? |
| NIST reports that current image generators underrepresent some groups in pictures of certain jobs. | ? |
| People may wrongly trust a model to work equally well for everyone. | ? |
| A model that works well in English must work well in every language. | ? |
Thoughtful work, reviewer. Tomorrow in the Explorer Lab you will look for sameness in SH-1's quizzes.