Raven breaks the seal on the envelope and fans out eight test cards. "No peeking at the labels yet."
Rocket covers the label column with a strip of paper. "Version 2 goes first. I will read the numbers."
Nova hovers over the score sheet. "Write each answer before you uncover the label," she says.
"That way nobody is tempted to change an answer."
Rocket slides the paper down one row. Raven reads the first card aloud.
They both lean forward to see what Sky Sorter says.
You will test Sky Sorter version 2 on eight held-back cards.
Then you will measure its accuracy and look closely at its mistakes.
SKY SORTER, VERSION 2 IF streak is yes THEN label = "satellite streak" ELSE IF size >= 8 THEN label = "comet" ELSE IF size >= 4 THEN label = "planet" ELSE label = "star"
| Card | Brightness | Size | Streak? | Version 2 label | True label |
|---|---|---|---|---|---|
| T1 | 130 | 2 | no | star | |
| T2 | 220 | 5 | no | planet | |
| T3 | 100 | 10 | no | comet | |
| T4 | 200 | 1 | yes | satellite streak | |
| T5 | 180 | 4 | no | planet | |
| T6 | 235 | 3 | no | star | |
| T7 | 150 | 7 | no | comet | |
| T8 | 90 | 3 | no | planet |
| Lab finding | True or false? |
|---|---|
| Version 2 scored lower on test cards than on training cards. | ? |
| Both mistakes involved cards with no streak. | ? |
| Version 2 got every satellite streak wrong. | ? |
Superb testing! Tomorrow we refine Sky Sorter and test it again, fairly.