← Back to course
AI and You 9-12 / Week 05 / Explorer Quest
6/6
Week 05 · Measuring a Model

Explorer Quest

Explorer Quest: Test the guesser
// The right test, the right numbers
⏱ about 20 min

Explorer Quest: Test the Guesser

Comet hides a wooden spoon in a shoebox and shakes it. "What is in the box?"

Wren listens. "A pencil."

"Wrong!" Comet says. "But I only shook it once. Was that a fair test?"

Wren frowns. "One question is not a test. And a spoon and a pencil sound alike in a box. Of course I mixed them up."

Nova hovers over the shoebox. "Would you like a hint?" she asks. "Write the answers down before anyone guesses, and pick objects that sound different."

Comet grins and hands you the box. "Five objects, a real answer key, and a score at the end."

  1. A fair test matches how the guesser will really be used.
  2. Write the answer key before anyone guesses.
  3. Accuracy is how many guesses are right, out of all the guesses.
◇ TEST THE GUESSER
You need a family member, a box with a lid (or a towel to cover a bowl), paper and a pencil.
Choose five small, safe household objects that are fine to shake.
For example: a sock, a spoon, a pencil, a plastic cup and a ball of paper.
The tester writes the five objects in order on paper as the answer key, and keeps it hidden.
The guesser closes their eyes. The tester puts the first object in the box, closes it and shakes it gently.
The guesser says what they think is inside. The tester writes the guess next to the key.
Repeat for all five objects.
Score it together: count how many guesses match the key. Each right guess out of five is 20 percent.
Switch roles and test again with five different objects.
Talk together: was the test fair? What would a fairer test need?
For a grown-up
This week your student learned how to test whether an AI system works for its real purpose.
They also learned about two kinds of mistakes: false alarms and missed errors.
Ask: why should the answer key be written before the guessing starts?
Ask: if all five objects sounded alike, would the score be fair?
Everything in this course is done on paper. No app, website, account or AI tool is needed.
QUEST CHECK
  • Read the question.
  • Tap your answer.
The guesser gets 4 of 5 right. What percent is that?
Which change makes the test fairer?
Quest reviewTrue or false?
The answer key should be written before anyone guesses.?
One shake of one object is enough to test a guesser.?
A test with five objects that all sound the same is a fair test.?
WHY THIS EXERCISEThe same rules make a fair test for a family guesser and for SH-1.
← Friday