Percentiles & IQR Explained Simply

Full transcript

Word for word what is said in the video — the same text as the subtitles. Times are minutes:seconds into the video.

0:00The answer that leaned on an assumption ▶

Last time, a height of two metres came out at z = 3.75 — about one in eleven thousand. But that answer leaned on something: it assumed a bell curve. So what do we do when the bell breaks?

0:18Not everything is a bell ▶

Plenty of things are not bell-shaped. Think of waiting times, or incomes, or how long people stay in hospital. They bunch up on the left and trail off to the right. Watch what that does: the long tail drags the mean along with it, while the median stays where the people are. In video one, the mean, the median and the mode all landed on the same spot — but that was a promise the normal distribution makes, and this shape doesn't make it. So we want a way to say how special something is that never asks what shape the data has.

1:01The percentile: sort, then count ▶

Here is the idea, and it needs no formula at all. Line everyone up, shortest to tallest, and count. If ninety percent of people are at or below you, you're at the ninetieth percentile. Sort, then count — that's the whole method. It uses only the order of the data, so it can't care what shape the data has. You have seen this before: a growth chart is nothing but percentile lines. And it needs no bell curve. Measure a thousand people, find that just one is taller than our man, and he sits at the 99.9th percentile. You didn't assume anything — you just counted.

1:49Q1, the median and Q3 ▶

Three of these percentiles are used so often that they got their own names. The 25th percentile is the first quartile. The 50th is the median — we already know that one. And the 75th is the third quartile. Together they cut the line- up into four equal groups. For our heights, the first quartile lands at 164.6 centimetres and the third at 175.4. So a quarter of people are shorter than 164.6, and a quarter are taller than 175.4.

2:30The IQR: the width of the middle half ▶

The distance between those two has a name: the interquartile range. Third quartile minus first quartile — 175.4 minus 164.6 is 10.8 centimetres, the width of the middle half of everyone. And notice what it ignores: the tallest person could grow another metre, and the shortest could shrink one, and this number would not move at all. That isn't a weakness — as we're about to see, it's the entire point.

3:06One typo, two toolkits — masking ▶

Back to our 250 people, where somebody now fat-fingers one measurement: two metres is typed as twenty metres. Watch the left-hand toolkit: the mean climbs from 170 to 177 centimetres, and the standard deviation goes from 8 to 116. And here is the damage — our real two-metre man had a z-score of 3.75. With that single typo sitting in the data his z-score drops to 0.2, and he now looks completely ordinary. The tool we use to find outliers has been blinded by an outlier, and statisticians call that masking. Now the right-hand side: the median is still 170, and the interquartile range is still 10.8. Neither of them flinched, because neither of them ever looked at anything but the order.

4:06In one picture ▶

So, in one picture: a percentile is your place in the line-up — sort, then count. The first quartile, the median and the third cut that line-up into four. The interquartile range is the width of the middle half. And where a single bad value can wreck the mean and sigma, the median and the IQR barely notice.

4:35What's next: the boxplot ▶

Next time, we turn all of this into a single picture: the boxplot. And tell me below — has one bad data point ever wrecked your average? Until then: happy learning.