← Back to course
AI and You 9-12 / Week 08 / Friday
5/6
Week 08 · Safety and Security

Friday

Impact Friday: Everyday testers
// Planted notes, poisoned cards and paper red teams
⏱ about 20 min

Friday: Impact Friday: Everyday Testers

Comet is stuck. "Red-teaming sounds like a job for experts. We are just students."

Librarian Joss smiles. "Who found the planted note? Who counted the poisoned cards?"

Wren holds up the RT-8 results. "And RT-7 came from Teacher Mara's idea. She knew the Spanish gap would matter."

Comet brightens. "So different people notice different things. A Spanish speaker, a Biology teacher, a student up late."

Nova projects the red-team list with a name beside each idea. "Would you like a hint?" she asks. "Look at who thought of each card."

Wren looks at you. "If SH-1 runs at Harbor Point, who keeps testing it after we are done?"

Everyday people as red-teamers

NIST says red-teaming can be done by the general public: everyday users who are not AI experts.

NIST also says diverse red teams, with people of different backgrounds and fields of study, can find flaws.

They can find flaws in the many different settings where a system will be used.

In the SH-1 story, the crew is that kind of team. A Spanish teacher, a Biology teacher and students each spotted different problems.

WHO SPOTTED IT?
  • Read the question.
  • Tap your answer.
Who was best placed to suggest testing a Spanish question in Spanish?
Why do diverse red teams find more flaws?

Rate R4: information security

Risk combines how likely a harm is with how big it would be. The crew rates R4 with this week's evidence.

Likelihood: Medium. RT-5 shows SH-1 can follow a planted note, but teachers can check pages before upload.

Size of harm: Medium. A false "test cancelled" or wrong library hours would confuse students, but a teacher can correct it.

RowRiskSH-1 exampleLikelihoodHarm
R4Information securityPlanted note in uploaded notes; poisoned notesMediumMedium
AI actor spotlight: everyday testers and operation and monitoring
NIST says general public red-teamers are everyday users who will use a system or its outputs.
Operation and monitoring work means running the system and regularly checking its output and effects, with others.
NIST lists system operators, domain experts, users who act on the output, evaluators and auditors, and researchers among these people.
At Harbor Point, teachers and students would do this work during a pilot.

The model card: security section

The crew adds its security plan to the SH-1 model card. These are the crew's choices in the story.

  • A teacher checks every notes page before SH-1 reads it.
  • SH-1 treats notes as data, not orders; the builder must show how.
  • Keep a record of SH-1's study material and who added each part.
  • Run the RT-8 cards again each term, with testers from every class.
  • A teacher can switch SH-1 off or step in at any time.

Week review

  1. Generative AI widens the attack surface: the AI itself can be attacked.
  2. Prompt injection changes the input; indirect prompt injection hides orders in data the system reads.
  3. Data poisoning tampers with training data. Three cards changed Tiny Text's answer.
  4. Red teams plan tests with a safe answer for each. Diverse teams find more flaws.
  5. Secure, resilient and safe ask different questions. Safety starts at design.
  6. R4 information security: likelihood Medium, harm Medium.
WEEK CHECK
  • Read the question.
  • Tap your answer.
NOTE-7 hid an order inside notes SH-1 would read. What kind of attack is that?
Three new cards changed Tiny Text's answer to "ten". What kind of attack is that?
Week checkTrue or false?
The crew rated R4 Medium for likelihood and Medium for harm.?
Only AI experts can be useful red-teamers.?
Operation and monitoring includes regularly checking a system's output.?
WHY THIS EXERCISEThe rating and the security plan are what the crew brings to the panel.
On paper, add the security section to your SH-1 model card. List the five defenses and circle the one you would do first.

Brilliant week, reviewer. Tomorrow is the Explorer Quest: a family game of spot the planted note.

← Thursday