Rocket shakes the word cups from last week and draws slips. "Raven spots a comet," he reads.
"That sounds great. I will add it to the club report."
Raven checks the real mission log. "Wait. I never spotted a comet. You did, twice."
"But the sentence sounded so sure," Rocket says. "There was no maybe in it at all."
Nova drifts over the cups, her lights flickering. "The cups do not know who spotted what," she says.
"They only know which words tend to follow which. So how sure should a sentence sound before you trust it?"
NIST's Generative AI Profile describes a risk it calls confabulation.
Confabulation is when generative AI confidently states something that is wrong or false.
People often call this a "hallucination" or a "fabrication".
NIST says confabulations are a natural result of how generative models are designed.
| Statement | True or false? |
|---|---|
| Confabulation means stating something false with confidence. | ? |
| If an AI answer sounds sure, it must be true. | ? |
| People often call confabulation a "hallucination". | ? |
| Confabulation is a natural result of how generative models are designed. | ? |
NIST notes that some people think "hallucination" is a misleading word.
It makes the AI sound like a person who sees things. Real AI is a tool that people build, not a mind.
NIST's Generative AI Profile uses the word confabulation for this risk.
Sharp start. Tomorrow you find out why prediction can land on a false sentence.