← Back to course
Statistics 9-12 / Week 09 / Thursday
4/6
Week 09 · Comparing Two Treatments

Thursday

When chance can explain it
// Two warm-ups, one shuffled deck
⏱ about 20 min

Thursday: When Chance Can Explain It

"Second experiment," Comet says. "Start cue. A whistle or a clap. The deck picks six and six again."

Laps are run. Nova reports: "Whistle mean 93.3, clap mean 93. Difference 0.3 seconds."

"Tiny," Wren says. "But small gaps can still matter. What do you notice when we shuffle?"

Twenty shuffles later: "8 of 20 reached 0.3 or more," Nova says. "That is 40%."

"Way over 5%," Comet says. "Chance makes a gap like ours almost half the time."

"So the cue made no difference we can see," Wren says. "The whistle and the clap are a tie, for these runners."

Nova glows. "Would you like a hint? Not significant is a real answer, not a failure."

"Then the Run Log says: warm-up, yes. Start cue, no," Comet says. "Both are useful to know."

Reading a difference that chance can explain

The start-cue gap was 0.3 seconds. In 20 shuffles, 8 reached it: 40%. That is far above 5%.

Chance alone makes a gap this big often. The crew cannot tell the cue apart from luck, so it says the difference is not significant.

Not significant does not mean the cue has zero effect. It means this experiment could not see one. A bigger experiment might.

ExperimentObserved differenceShuffles at or beyondPercentJudgment
warm-up: stretch minus jog4 seconds1 of 254%significant
start cue: whistle minus clap0.3 seconds8 of 2040%not significant
A dot plot of 20 shuffled differences from -4 to 3, with the crew's 0.3 marked near the middle.
THE START-CUE EXPERIMENT
  • Read the question.
  • Tap your answer.
Whistle group: 93, 95, 90, 96, 92, 94 seconds. Clap group: 94, 91, 96, 92, 95, 90 seconds. What is the difference of the means, whistle minus clap? (Round to 1 place.)
In 20 shuffles, 8 gave a difference of 0.3 or more. What percent of the shuffles is that? (Round to 1 place.)
In 20 shuffles, 8 gave a difference of 0.3 or more. Is the start-cue difference significant by the 5% rule?

How to re-randomize with cards

The crew's procedure, written out. Read it first, then put it in order.

  1. Write every runner's result on its own card, all groups together.
  2. Shuffle the whole deck well, so the treatment labels are forgotten.
  3. Deal two hands with the same sizes as the real groups.
  4. Find the mean of each hand and record the difference, first minus second.
  5. Repeat many times and plot every difference on one dot plot.
  6. Count the shuffles at or beyond the observed difference, divide by the total, and compare with 5%.
RE-RANDOMIZING WITH CARDS, IN ORDER
  • ?Write every result on its own card
  • ?Shuffle the whole deck well
  • ?Deal two hands the same sizes as the real groups
  • ?Count the shuffles at or beyond the observed gap and compare with 5%
  • ?Find each hand's mean and record the difference
  • ?Repeat many times and plot every difference
WHY THIS EXERCISEThe procedure is the same for every experiment. Only the cards change.
A randomized experiment compares two of these. Type one word.
What percent of the start-cue shuffles reached 0.3 or more? Type the number.
WHICH CONCLUSION IS JUSTIFIED?
  • Read the question.
  • Tap your answer.
The warm-up gap of 4 seconds was significant. Which conclusion is justified?
The start-cue gap of 0.3 seconds was not significant. Which conclusion is justified?
Runners who chose the jog themselves ran faster than runners who chose to stretch. What is justified?
StatementTrue or false?
8 ÷ 20 × 100 = 40?
Not significant means the treatment is proven to have no effect.?
A significant result points to a cause only because the groups were randomly assigned.?
The re-randomizing procedure changes when the treatments change.?
WHY THIS EXERCISEKnowing what each judgment does not say keeps the crew honest when the result is exciting.

Careful thinking. Tomorrow you take the method into everyday comparisons and review the week.

← Wednesday