← Back to course
Statistics 9-12 / Week 09 / Tuesday
2/6
Week 09 · Comparing Two Treatments

Tuesday

What chance alone can do
// Two warm-ups, one shuffled deck
⏱ about 20 min

Tuesday: What Chance Alone Can Do

"Suppose the warm-up made no difference at all," Wren says. "Then each runner's time is just that runner's time."

"And the red-blue split was pure chance," Comet says. "So any split is as likely as the one we got."

Wren writes the 12 lap times on 12 cards. Shuffle. Deal six and six. "Difference: -5."

Again. -4. Again. 0. Nova logs each one. Twenty-five shuffles later a dot plot has grown.

"What do you notice?" Wren asks. "Most shuffles land between -3 and 3. Only 1 reached 4 or more."

"1 of 25 is 4%," Comet says. "Chance alone rarely makes a gap as big as ours."

Nova glows. "Would you like a hint? The crew's rule says under 5% is significant."

"Then the jog really did help our runners," Comet says. "Chance is not a good enough explanation."

Way 1: re-randomize with cards

If the treatment did nothing, the lap times belong to the runners, not to the groups. Any shuffle of the cards is a split chance could have made.

Shuffle all 12 time cards, deal 6 and 6, and find the difference of means. Record it. Repeat many times.

The dot plot of shuffled differences shows what chance alone does. Then the crew asks where its observed difference lands.

Shuffle12345678910111213
Difference-5-4-3-3-3-2-2-1-1-1000
Shuffle141516171819202122232425
Difference001111223334
A dot plot of 25 shuffled differences from -5 to 4, with the crew's 4 marked in red.

Way 2: the 5% rule

Count the shuffles at or beyond the observed difference. Here 1 of 25 reached 4 or more: 4%.

Under 5%: chance alone rarely gives a gap this big, so the difference is significant. The treatment is the best explanation.

5% or more: chance alone could easily give this gap. The crew cannot tell the treatment apart from luck.

The picture and the percent say the same thing. The dot plot shows it; the percent names it.

READ THE SHUFFLES
  • Read the question.
  • Tap your answer.
In 25 shuffles, 1 gave a difference of 4 or more. What percent of the shuffles is that? (Round to 1 place.)
In 25 shuffles, 1 gave a difference of 4 or more. Is the crew's observed difference significant by the 5% rule?
The 25 shuffled differences are in the tables above. What is their median?
The 25 shuffled differences are in the tables above. What is their range?
TWO WAYS, ONE IDEA
  • Read the question.
  • Tap your answer.
What does each shuffle of the time cards pretend?
The crew's 4 sits at the far right edge of the dot plot. What does that show?
Shuffling every result into two new groups is called re-what? Type one word.
By the crew's rule, a difference is significant when fewer than what percent of shuffles reach it? Type the number.
StatementTrue or false?
1 ÷ 25 × 100 = 4?
A significant difference means chance alone rarely produces one that big.?
Significant means the treatment is proven to work for every runner everywhere.?
More shuffles make the dot plot a better picture of chance.?
WHY THIS EXERCISEThe 5% rule is a line the crew agreed on in advance, so the judgment is not bent to fit the hope.
Try it
Shuffle your 12 time cards from Monday and deal six and six. Find the difference of means, first hand minus second.
Do it five times and mark each difference on a number line from -6 to 6. Where does 4 sit?
Draw the dot plot of the 25 shuffled differences from -6 to 6 and mark 4 with a red line.

Two ways that agree. Tomorrow is Data Lab: your hands shuffle, your dot plot grows, and you judge the gap yourself.

← Monday