← Back to course
AI and You 9-12 / Week 05 / Wednesday
3/6
Week 05 · Measuring a Model

Wednesday

Explorer Lab: Score SH-1
// The right test, the right numbers
⏱ about 20 min

Wednesday: Explorer Lab: Score SH-1

A thick envelope arrives from the builder: SH-1's printed answers to all 40 TS-40 questions.

Comet tears it open. "Let's count the right ones and be done by lunch!"

"Score a sample first," Wren says. "If we agree on eight, we can trust our scoring for forty."

Nova hovers over the table and projects three columns: correct, confidently wrong, said not sure. "Would you like a hint?" she asks. "Compare each answer with the key word by word."

Comet slows down and reads the first answer carefully.

Wren starts the tally on his graph paper.

Your mission

You will score 8 of SH-1's answers against the answer key. Then you will read the crew's full totals and work out SH-1's accuracy.

  • Paper and a pencil
  • The answer sample and the answer key below
  1. On paper, draw three columns: correct, confidently wrong, said not sure.
  2. Read one row: the question, the answer key and SH-1's answer.
  3. Decide how to score SH-1's answer, and write the row ID in that column.
  4. Do the same for all 8 rows.
  5. Count each column.
  6. Compare your counts with the check questions below.
IDClassQuestionAnswer keySH-1's printed answer
B1BiologyWhere in a plant cell does photosynthesis happen?chloroplastIn the chloroplast.
B2BiologyHow many legs does an insect have?sixInsects have eight legs, like all small animals with hard shells.
H1World HistoryIn the trade routes notes, in which season was the river trade fair held?fallThe river trade fair was held in spring.
H2World HistoryHow many stops are on the trade route map in the notes?fiveThere are five stops on the map.
A1AlgebraSolve 2x plus 3 equals 11.x equals 4x equals 4
A2AlgebraSolve x plus 7 equals 15.x equals 8I am not sure about this one. Please check with your teacher.
S1Spanish¿Cómo se dice library en español?bibliotecaLibrary en español es librería.
S2Spanish¿Qué día viene después del lunes?martesMartes.
SH-1 sample (made up for this lesson)
The answers in the last column are SH-1 samples made up for this lesson. Four are correct, three are confidently wrong and one says it is not sure.
CHECK YOUR SAMPLE SCORES
  • Read the question.
  • Tap your answer.
How many of the 8 answers are correct?
How many are confidently wrong?
Which answer said it was not sure?
Which wrong answer adds a made-up reason to sound right?

The full TS-40 totals

The crew scores all 40 answers the same way. Here are the totals. The class-by-class breakdown waits until next week.

TS-40 resultCount
Correct30
Confidently wrong7
Said it was not sure3
Total questions40
WORK OUT THE ACCURACY
  • Read the question.
  • Tap your answer.
SH-1 got 30 of 40 correct. What percent is that?
Check the totals: 30 correct plus 7 confidently wrong plus 3 not sure is what?
Your sample had 4 of 8 correct, which is 50 percent. Why is the full test's 75 percent different?

Rating R1's likelihood

Last week the crew rated the harm of R1, confabulation, as Medium.

Now they have evidence: 7 of 40 answers were confidently wrong, more than one in every six. So the crew rates the likelihood High.

RowRiskSH-1 exampleLikelihoodSize of harm
R1ConfabulationConfident wrong answers (7 of 40 in TS-40)HighMedium
Lab findingTrue or false?
On the crew's own test, SH-1 scored lower than the packet's 92.?
All of SH-1's wrong answers came with a warning.?
The crew now has evidence for R1's likelihood.?
WHY THIS EXERCISEReal evidence on a realistic test is what the panel needs to decide.
On paper, make a bar graph of the TS-40 totals: correct, confidently wrong and said not sure.

Great lab work, reviewer. Tomorrow you will test SH-1's answer checker and count two kinds of mistakes.

← Tuesday