Comet bursts in with a plan. "Easy fix! The builder adds more Spanish material, the score goes up, and SH-1 is fair."
Wren tilts his head. "Fair for whom? What if a student cannot use SH-1 at all?"
Teacher Dev looks up. "And what did the builder assume about our school in the first place?"
Comet flips to the packet. "It says SH-1 was tuned on study material. It never says which languages."
Nova hovers over the packet and highlights one word: every. "Would you like a hint?" she asks. "Ask what the builder believed about its users."
Comet chews her pencil. "They assumed every class is taught in English."
Fairness is hard to define, because people and cultures see it differently.
Fixing harmful bias does not make a system fair by itself.
A system may still be hard to use for people with disabilities, or for people without good access.
One kind of bias is a wrong assumption developers make about their users.
Model collapse can happen when a model is trained too much on synthetic, AI-made data.
Some kinds of data disappear from what the new model produces.
Here is a made-up example in words. A quiz model is trained on the teachers' quizzes. Then it is retrained on its own quizzes, three times over.
| Version | Trained on | Kinds of question it writes |
|---|---|---|
| Version 1 | The teachers' quizzes | 5 |
| Version 2 | Version 1's quizzes | 3 |
| Version 3 | Version 2's quizzes | 1 |
| Version 4 | Version 3's quizzes | 1 |
This table is a made-up example to show the idea. It is not a measurement of SH-1 or any real model.
| Statement | True or false? |
|---|---|
| Fixing harmful bias always makes a system fair. | ? |
| People and cultures can see fairness differently. | ? |
| A wrong assumption about users is one kind of bias. | ? |
| Training too much on AI-made data can make some outputs disappear. | ? |
Deep thinking, reviewer. Tomorrow is Impact Friday: what to do about the Spanish gap.