Law of Small Numbers (Sample Size Neglect)

Class Selection Problem

It is the first week of the winter term. Henry has signed up for five classes, but plans to take only four. Three of these classes he knows he wants, so he must decide between the other two. Both are on subjects interesting to him. The student course evaluations indicate that Course A is better taught. However, he attended the first meeting of both classes this week, and found Course B’s session more enjoyable. Which class should he take and why?

The excerpt also compares a one-game championship with a multi-game series, though the full championship question is not included. Subjects wrote open-ended answers to each type of problem. These were coded as to whether they reflected the use of statistical principles – chiefly, the law of large numbers or the regression principle – or not. An example of a nonstatistical answer for the championship problem is the following: “Super Bowl, because of neutral ground and also a one shot deal. Either you make it or break it – one chance. The pressure is on to perform the team’s best.” An example of a statistical answer is the following: “World Series is better. Anyone can get lucky for one game, but it is harder to be lucky for four. Besides, being home or away is part of the game, you don’t play on neutral ground during the season.” An example of a nonstatistical answer for the class-choice problem is the following: “He’s got to choose for himself.” An example of a statistical answer is: “You can’t tell from one time – thus a survey that is over a longer range is better. Although Henry’s idea of a good class could be different from most students.”

Statistical answers were much more common for problems about objective events than for problems about subjective events. Forty-one percent of the answers for the former were statistical; the range of mean percentage statistical answers across problems was 30% to 93%. Only 12% of the answers to subjective problems were statistical; the range was 5% to 16%.

The results also showed that subjects were consistent in their tendency to give statistical answers. Those who gave statistical answers for any given problem were more likely to give them for any other. This tendency was correlated with both verbal and mathematical scores on the Scholastic Aptitude Test.