Rosa wants a chatbot to write birthday messages for her regular customers. She opens her customer list to paste it in.
It has names, phone numbers, home addresses and notes about who orders flowers for a sick relative.
Her cursor hovers over the paste button. Then she stops.
Where would that list go once she pasted it? And who else might ever see it?
NIST lists data privacy as one of the risks of generative AI.
The risk is that health, location, biometric or other personal information leaks, or is used in ways no one agreed to.
Biometric information comes from the body, such as a face, a fingerprint or a voice.
Personal information, often called PII, means details that can identify a person. PII stands for personally identifiable information.
NIST also notes that language models have revealed sensitive information that was in their training data.
This problem is called data memorization. Information a model learned from can, in some cases, come back out in an answer.
This course does not name tools or say how any one tool stores what you type. Read your own tool's privacy settings.
NIST warns that generative AI tools can be attacked, for example by prompt injection or data poisoning.
Prompt injection hides instructions in text a tool reads, to make it do something its user did not intend.
Data poisoning slips bad or false examples into the data a tool learns from.
You do not need to defend against these yourself. They are one more reason to keep private details out.
The US Small Business Administration advises businesses to try not to feed sensitive data or proprietary information into AI tools.
Proprietary information is what belongs to your business and gives it an edge, such as prices, plans or supplier lists.
The SBA says this helps keep that information out of the data pool the system uses to produce content.
| Statement | True or false? |
|---|---|
| A voice or a fingerprint is biometric information. | ? |
| NIST says AI tools can be attacked by prompt injection. | ? |
| Data poisoning means a tool gets too much data. | ? |
| Rosa's customer list passes the notice board test. | ? |
Sources: NIST AI 600-1, Generative AI Profile (2024) · US Small Business Administration, AI for small businesses
Tomorrow: why children's personal information has special legal protection.