Comet rushes in with a new idea. "Models are great! Let's build one that decides who gets the workbench first."
She shows her rule: members who visited most last week go first. She used the visit counts from the survey.
Wren reads the rule twice. "What do you notice about who is in this data?"
Comet frowns. "Only the 40 members who answered. And M17 visited 23 times, so the volunteer always wins."
"And the 80 members who never answered?" Wren asks.
Nova hovers above the rule. "Would you like a hint?" she asks. "A model can only see what you put into it. Who is invisible here?"
Comet sets down her pencil. "Data analyst, help me check my own rule."
Computing innovations can reflect human biases, through their algorithms or through their data.
Bias can enter at every level of building something. Programmers should act to reduce it.
A model can also carry bias from what was left in or left out.
| Comet's workbench rule (a made-up example) | What it means |
|---|---|
| Uses only the 40 survey answers | The 80 members who did not answer are ranked last |
| Ranks by visits last week | M17, the volunteer with 23 visits, always goes first |
| Leaves out why people visit | A member who needs the bench once a week can never get it |
Innovations are often used in ways their makers never intended.
Data mining has helped medicine and science. It has also been used to discriminate against groups of people.
Responsible programmers think about unintended uses and their effects. Still, no one can foresee every use.
| Statement | True or false? |
|---|---|
| Computing innovations can reflect human biases through their data. | ? |
| Bias can enter only at the last step of building a model. | ? |
| Data mining has been used to discriminate against groups of people. | ? |
| A careful programmer can foresee every way a model will be used. | ? |
Thoughtful work this week, data analyst. Tomorrow is Mission Quest: model a line at home.