applets.

Average Trap

← All apps

About Average Trap

One number can't describe a whole set of numbers, and the one you are given may not be the one you need. Four ways it goes wrong, one tab each.

AVERAGES: the data are brass weights on a rule. Drag one (or click it and use ← →) and the mean, median, mode, range, standard deviation and interquartile range all follow. ADD ONE and TAKE ONE OFF change the set; RESET puts the example back. On a big screen a second rule above shows what happens if it rests on the median instead.

BOX PLOT: the same data drawn as a box plot under a histogram. You can drag the weights here too, and the Bars slider changes how the histogram is cut up (that changes its look a lot). SAME BOX shows three sets that give exactly the same box plot; press SHOW WHAT'S INSIDE to see how different they are. THE ORIGINAL is the two box plots the Python script box_plot.py drew.

SAME STATS: Anscombe's quartet, and a scatter of your own: drag its dots and watch the correlation r and the best-fit line.

SIMPSON: one option wins in every group (or nearly every one), the other wins when the groups are added together. Each circle sits at a group's rate, sized by how many it stands for; COMBINE THE GROUPS rolls each side's circles down into one.

"AVERAGE" MEANS SEVERAL THINGS

The mean is the total shared out equally: add them up and divide by how many. It is also the balance point, which is why the rule here rests on a knife-edge at the mean: a weight a long way out has a lot of leverage. The median is the middle value when they are put in order (half below, half above), so a single extreme value barely moves it. The mode is the value that turns up most often, the only "average" that works for things like shoe sizes or favourite colours. When a news story says "average" it usually means the mean; UK official figures for pay quote the median, because a few very high earners pull the mean up.

Spread matters as much as the middle. The range is largest minus smallest. The interquartile range (IQR) is the spread of the middle half. The standard deviation is roughly the typical distance from the mean: square each distance, average the squares (dividing by one less than the count), take the square root. That is the sample standard deviation, for a sample from something bigger; for a whole population you divide by the count itself.

THE STORY

Karl Pearson named the standard deviation in 1894 and the mode in 1895. The box plot is John Tukey's, from his 1977 book Exploratory Data Analysis; Mary Eleanor Spear had drawn a similar "range bar" in 1952. Tukey's whiskers reach the furthest values within 1.5 IQR of the box, and anything further out is drawn as its own dot.

In 1973 the statistician Francis Anscombe published four small data sets, eleven points each, that share the same means, the same variances, the same correlation (0.816) and the same best-fit line (y = 3.00 + 0.500x), yet look nothing alike when drawn: a loose line, a curve, a straight line with one stray point, and a single far point doing all the work. His point was simple: draw the data before trusting the numbers. In 2017 Justin Matejka and George Fitzmaurice took the idea further with the "Datasaurus Dozen", thirteen sets with the same statistics, one of them a dinosaur.

The reversal now called Simpson's paradox was noticed by Karl Pearson in 1899 and Udny Yule in 1903; Edward Simpson described it in 1951 and Colin Blyth gave it the name in 1972. The two real examples here are famous ones. In 1986 C. R. Charig and colleagues compared open surgery with keyhole surgery (percutaneous nephrolithotomy) for kidney stones: open surgery did better for small stones and for large stones, yet keyhole surgery looked better overall, because the difficult large stones mostly got open surgery. In autumn 1973 the University of California, Berkeley admitted 44% of the 8,442 men who applied for graduate study and 35% of the 4,321 women. Peter Bickel, Eugene Hammel and William O'Connell (Science, 1975) looked department by department and found no bias against women, if anything a small one in their favour: women had mostly applied to the departments that turned most people away. The page uses the six largest departments, as tabled in Freedman, Pisani and Purves' textbook Statistics. The exercise example is invented to show the same thing on a scatter plot.

HOW IT WORKS

Everything is worked out afresh from the numbers each time a weight moves. Quartiles here are the medians of the lower and upper halves (with an odd count the middle value goes in neither half); other books use slightly different rules, which can shift a quartile a little. The histogram counts how many values fall in each bar, the way the Python script Bar Chart.py drew its fruit counts. Correlation is Pearson's r and the line is least squares. In Simpson's paradox each overall rate is a weighted average of the group rates, weighted by how many are in each group: it is where that side's circles would balance on a rule.

The salaries, house prices, shoe sizes and test marks are invented but typical.

FURTHER WATCHING AND READING