Rocket is sorting a big stack of new snapshots. Sky Sorter calls out a label, and he copies it into the log.
Card after card, he stops looking at the snapshots at all.
Raven picks up one card. "Sky Sorter said star. This one has a long, faint tail."
"But the machine is usually right," Rocket says. "Why would I double-check?"
Nova floats between them, her lights a careful amber. "Usually is not always," she says.
"When did you stop being the one who checks?"
Rocket looks down at the stack and slowly pulls the first card back out.
NIST describes automation bias as trusting automated systems too much.
People may over-rely on generative AI. They may even think its content is better than content from other sources.
Last week you met human-cognitive bias. Automation bias is an example of how people read and use output.
NIST also lists treating AI as if it were a person as a risk. The word for that is anthropomorphize.
Emotional attachment to AI is another risk NIST names.
Nova is a story character, so she can care about the crew. Real AI is a tool people build. It does not think or feel like a person.
| Statement | True or false? |
|---|---|
| Automation bias means trusting automated systems too much. | ? |
| If a tool is usually right, you never need to check it. | ? |
| Real AI tools think and feel like people. | ? |
| Emotional attachment to AI is a risk NIST names. | ? |
Thoughtful work. Tomorrow you build a checking habit for everyday life.