Rigged or Random?: teacher guide
Statistics, Algebra 2, Biology or Chemistry (experimental design), Journalism or Media Literacy. Grades 9–12. One class period.
What students will be able to do
- Explain why genuinely random sequences contain long streaks, and why people faking randomness avoid them.
- Use a count (switches, longest run) to argue whether a sequence is likely random, and say how confident that argument is.
- Explain regression to the mean using the free-throw example, and recognize it in sports and school data.
- State what p < 0.05 means and why testing many things produces false positives (the multiple comparisons problem).
- Explain why replication is the test of a real effect.
Before class
Play Act 1 yourself and get caught. Almost everyone does. That is the opening of the lesson: the teacher got caught too. The game takes 15 to 20 minutes for most students; the debrief carries the rest of the period.
A 45-minute plan
| Time | What happens |
|---|---|
| 0–5 | Hook, no screens. Half the class writes 40 real coin flips on paper (flip a coin or use a phone). The other half writes 40 made-up flips. Collect a few from each and read them aloud. Can the class tell which is which? Usually not yet. |
| 5–12 | Act 1: Fake It. Students type a sequence and get judged. Ask: what did the detector count? (Switches and longest streak.) Go back to the paper sequences from the hook: now can they tell? |
| 12–18 | Act 2: Spot the Fake. Eight rounds. Keep score as a class. The tell: streaks of 4 or more. |
| 18–26 | Act 3: The Lucky Streak. Everyone picks a shooter, then game day. Collect: how many students' picks did worse on game day? Nearly all. Name it: regression to the mean. |
| 26–35 | Act 4: The Jelly Bean Study. Students run the study several times and replicate. Tally the class's false positives per study; it lands near 1 in 20 per color tested. |
| 35–45 | Debrief with the questions below. |
Discussion questions
- A basketball announcer says a player who hit five in a row "has the hot hand." What would you need to see to believe that?
- Your school's test scores jumped last year and dropped this year. The principal wants to know what went wrong. What should you say?
- A headline says "Study finds chocolate linked to better grades." What three questions do you ask before believing it?
- In Act 4 the scientists were honest. Every test was done correctly. How did they still end up with a false headline?
- Act 1's detector sometimes calls a real coin "human." How often? Is that acceptable? (It uses the same 5% rule as the jelly bean study. Connect the two.)
- Where in your own life have you seen a pattern in something that was probably random?
Misconceptions to listen for
- "Random means evenly mixed." It means unpredictable. Evenly mixed is a pattern, and it's the pattern humans produce when they fake it.
- "After five heads, tails is due." The gambler's fallacy. The coin has no memory. Act 1's real-coin button shows runs of 6 and 7 happening with no correction afterward.
- "The top scorer at tryouts is the best player." On a noisy measure, the top score is partly skill and partly luck. Only the luck part regresses, but you can't see which part is which from one measurement.
- "p < 0.05 means there's a 95% chance the effect is real." It means that if there were no effect, data this extreme would show up less than 5% of the time. Twenty honest tests of nothing yield about one "finding."
- "Replication failed, so the first scientists cheated." Not necessarily. Flukes are expected. That's exactly why we replicate.
Extensions
- Statistics: derive the expected number of switches in n flips (n−1)/2 and its standard deviation √(n−1)/2. Where does the detector's 5% line fall for n = 40?
- Compute by hand the probability that 40 flips contain no run of 4 or more. (Harder than it looks; simulation is a fine answer.)
- Media literacy: find a recent "new study finds" headline. How many things did the study test? Was it replicated?
- Biology or Chemistry: design the jelly bean study properly. What would pre-registration and a correction for multiple comparisons (for example Bonferroni: use 0.05 ÷ 20) change?
About the numbers in the game
The detector runs an exact two-sided binomial test on the number of switches (n−1 = 39 trials, p = 0.5) and compares the longest run to a simulated distribution of 20,000 random sequences. It flags "human" at the 5% level for switches or when the longest run is shorter than 90% of random sequences, so it wrongly flags a real coin roughly one time in ten; that is deliberate and worth discussing. The fakes in Act 2 come from a model that switches sides 62–72% of the time, which matches what people do when asked to write random sequences. Act 3 shooters all have exactly a 50% chance per shot. Act 4 p-values are drawn uniformly between 0 and 1, which is what honest tests produce when there is no true effect.