Librarian Joss brings three printouts from the builder. "SH-1 answered some test questions about our school. Tell me what you think."
Comet reads the first one. "It even gives a page number! That has to be right."
Wren walks to the shelves and checks the catalog. Then he checks it again.
"There is no book with that name," he says. "Not here, and not in the school office."
Nova hovers over the printouts. "Would you like a hint?" she asks. "Look for the parts that make an answer look trustworthy. Then check if those parts are real."
Comet slowly circles the page number. "So the proof itself could be made up."
Confabulation matters most for open-ended questions that need long answers.
It also matters most for topics that need expert knowledge.
The risk is that people believe false content because it sounds confident, and then act on it.
The answer key: the library has 3 quiet rooms, and there is no Harbor Point Building Guide. In Answer C, x should be 5.
A generative AI answer can include made-up reasoning steps or citations. They make a wrong answer look right.
Language models sometimes show tidy steps even when the final answer is wrong.
A language model could even falsely claim to be human.
Made-up falsehoods are mostly a problem in text. For images, audio or video, making up something not real can be exactly what the user wants.
Some people say "hallucination" is a misleading word for confabulation. It treats AI as if it were a person, and that is itself a risk.
| Statement | True or false? |
|---|---|
| A citation in an AI answer proves the answer is right. | ? |
| Confabulation is a bigger risk for long, open-ended answers. | ? |
| A language model could falsely claim to be human. | ? |
| Some people think "hallucination" makes AI sound too much like a person. | ? |
Careful checking, reviewer. Tomorrow is Impact Friday: what happens when people act on a confident wrong answer.