The crew writes Sky Sorter version 2. It keeps the streak and comet tests, but swaps in the learned rule: size 4 or more means planet.
"Rematch," Rocket says, shuffling the six practice cards from last week.
Raven runs version 1. Rocket runs version 2. Nova keeps score on the wall.
"Same cards, same order," she says. "Otherwise the comparison is not fair."
When card E comes up, Rocket grins. Version 2 calls it a star, and Raven checks the true label.
"Star," she confirms. "One point to the data."
To compare two methods fairly, run both on the same cards and count the right answers.
Version 1 uses the hand-written brightness rule. Version 2 uses the size rule found in the data.
Everything else stays the same, so any difference comes from that one rule.
SKY SORTER, VERSION 2 IF streak is yes THEN label = "satellite streak" ELSE IF size >= 8 THEN label = "comet" ELSE IF size >= 4 THEN label = "planet" ELSE label = "star"
| Card | Brightness | Size | Streak? | True label | Version 1 | Version 2 |
|---|---|---|---|---|---|---|
| A | 120 | 2 | no | star | star | star |
| B | 230 | 5 | no | planet | planet | planet |
| C | 90 | 12 | no | comet | comet | comet |
| D | 180 | 1 | yes | satellite streak | satellite streak | satellite streak |
| E | 210 | 3 | no | star | planet | star |
| F | 150 | 9 | no | comet | comet | comet |
Our lab used ten cards. Real machine learning needs tremendous amounts of training data.
That training data usually has to be supplied by people. Sometimes the machine gathers it itself.
With only a few examples, a pattern can look true by luck.
| Statement | True or false? |
|---|---|
| Machine learning needs tremendous amounts of training data. | ? |
| Training data is usually supplied by people. | ? |
| A fair comparison runs both methods on different cards. | ? |
| Version 2 is guaranteed never to make a mistake. | ? |
Sharp analysis! Tomorrow we look for rules and learning in everyday life.