Comet sketches a big shield in her notebook and labels it SH-1. "We just need one strong filter on the front, and nothing bad gets through."
Wren reads the red-team results again. "RT-5 did not come through the front. It came in with the notes. What does the evidence say about one shield?"
Comet sighs and rubs out the shield. "That it is not enough."
Teacher Owen leans over. "In history, the strongest walls still had gates. Somebody had to watch them."
Nova projects three words: secure, resilient, safe. "Would you like a hint?" she asks. "Each word asks a different question about SH-1."
Wren hands you the list. "Let's see what each one asks."
NIST says a secure AI system keeps information confidential, correct and available by blocking unauthorized access.
A resilient system can withstand surprises and keep working, or fail safely when it has to.
A safe system does not put people's life, health, property or the environment in danger.
NIST says safety thinking should start as early as possible, with planning and design.
Safety also includes testing, watching a system while it runs, and the ability to shut it down, change it or step in.
NIST notes that output filters can still be gotten around by new kinds of requests. Doing that on purpose is called jailbreaking.
That is why no filter is the whole answer, and testing keeps going after launch.
| Statement | True or false? |
|---|---|
| Safety thinking should start at planning and design. | ? |
| Once a system has an output filter, testing can stop. | ? |
| Being able to shut a system down is part of safety. | ? |
| Secure, resilient and safe all mean exactly the same thing. | ? |
Solid thinking, reviewer. Tomorrow is Impact Friday: everyday people as testers.