Rocket runs Sky Sorter version 2 on the ten lab cards from last week. "Ten out of ten! It is perfect!"
Raven raises an eyebrow. "Those are the cards we used to choose its cut-off. Of course it gets them right."
"That is like studying the exact quiz questions," Nova says. "You ace the quiz, but did you learn the subject?"
Rocket thinks about that. "So we need cards it has never seen."
Raven pulls eight labeled cards out of a sealed envelope. "I held these back from the start."
A fair check uses cards the sorter never learned from.
So the crew splits its labeled cards into two piles before building anything.
The training set is used to find patterns. The test set is held back and used only for checking.
If you test on training cards, the score can look much better than it really is.
| Pile | Used for | Can it change the sorter? |
|---|---|---|
| Training set | Finding cut-offs and patterns | Yes |
| Test set | Checking how well the sorter works | No, never |
Accuracy measures how often the sorter is right.
Accuracy equals the number right divided by the total number tested.
For example, 8 right out of 10 is 8 divided by 10, which is 0.8, or 80 percent.
accuracy = number right / total tested percent = accuracy x 100 Example: 8 right out of 10 accuracy = 8 / 10 = 0.8 percent = 0.8 x 100 = 80
| Statement | True or false? |
|---|---|
| The test set is held back while the sorter is built. | ? |
| Testing on training cards gives a fair score. | ? |
| Accuracy is the number right divided by the total. | ? |
| It is fine to change cut-offs to fit the test cards. | ? |
Excellent thinking! Tomorrow in the Explorer Lab, you open the envelope and test Sky Sorter.