Rosa asks a chatbot for a short description of her spring bouquet. The answer reads well. It is warm, friendly and confident.
Then she notices one line. It says every bouquet comes with free delivery across the state.
Her shop has never offered that. Where did the line come from?
Generative AI is the kind of AI that makes new content, such as text, images, audio or video. NIST describes it as modeled on the data it learned from.
Text tools built on large language models work by predicting the next word in a sentence, again and again.
NIST notes that this kind of prediction can produce accurate answers. It can also produce answers that are wrong or that contradict themselves.
Think about Rosa's bouquet. Suppose many shop descriptions in its training data mention delivery.
A tool predicting likely words may add a delivery line because it fits the pattern, not because it is true for Rosa.
Read this example. It was written for this lesson, and it contains two claims Rosa never gave the tool.
| Line in the output | Did Rosa give the tool this fact? |
|---|---|
| The bouquet has tulips, daffodils and eucalyptus. | ? |
| The florists are award-winning. | ? |
| Delivery is free anywhere in the state. | ? |
For anything a tool writes for you, ask one question about each line. Did I give it this fact, or did it come from a pattern?
Lines that came from a pattern are not always wrong. But they are not checked, so you check them.
Sources: NIST AI 600-1, Generative AI Profile (2024)
Tomorrow: what this means for your own track, at home, in class or at work.