Skip to content

The normal distribution

Probability · level 8, lesson 3 of 7

A question to start

A machine fills cereal boxes. The label says 500 g, and the machine is set to put in 505 g on average, but every box is a little different: the fill amounts follow a bell-shaped curve with a standard deviation of 3 g.

About what fraction of boxes are underweight (less than 500 g)? And if the company wants only 1 box in 100 to be underweight, what average should the machine be set to?

These questions have exact answers, and the same method answers questions about exam scores, measurement errors, heights, and the averages of anything measured many times. The curve behind them all is the normal distribution, the most important distribution in probability.

The idea

The bell curve

A continuous random variable \(X\) has a normal distribution with mean \(\mu\) and standard deviation \(\sigma\), written \(X \sim \operatorname{N}(\mu, \sigma^2)\), if its density is the bell curve

\[f(x) = \frac{1}{\sigma\sqrt{2\pi}}\, e^{-\frac12 \left(\frac{x - \mu}{\sigma}\right)^2}.\]

(Some books write \(\operatorname{N}(\mu, \sigma)\) with the standard deviation instead of the variance. Always check which.)

You will not need to work with this formula directly, but each part of it says something:

  • the square \(\left(\tfrac{x - \mu}{\sigma}\right)^2\) makes the curve symmetric about \(\mu\), highest at \(\mu\);
  • \(e^{-(\dots)^2}\) makes it fall away very fast: values more than 3 standard deviations from the mean are rare;
  • \(\sigma\) sets the width: double \(\sigma\) and the curve becomes twice as wide and half as tall;
  • the factor \(\tfrac{1}{\sigma\sqrt{2\pi}}\) makes the total area exactly 1. That \(\pi\) appears in a probability curve is a surprise; it comes from the famous integral \(\int_{-\infty}^{\infty} e^{-x^2}\,dx = \sqrt{\pi}\).

The curve bends from "curving down" to "curving up" exactly at \(\mu - \sigma\) and \(\mu + \sigma\), so you can see the standard deviation in the picture.

The 68-95-99.7 rule

Whatever \(\mu\) and \(\sigma\) are, the areas within a whole number of standard deviations of the mean are always the same:

μ−3σ μ−2σ μ−σ μ μ+σ μ+2σ μ+3σ about 68% about 95% about 99.7%
  • about 68% of the probability lies within 1 standard deviation of the mean (more precisely 0.6827);
  • about 95% within 2 (0.9545);
  • about 99.7% within 3 (0.9973).

So for the cereal boxes, about 95% weigh between \(505 - 6 = 499\) g and \(505 + 6 = 511\) g.

The standard normal and z-scores

Every normal curve is the same shape, just shifted and stretched. So we only need a table for one of them: the standard normal \(Z \sim \operatorname{N}(0, 1)\), with mean 0 and standard deviation 1.

To move any normal \(X\) onto it, compute the z-score

\[Z = \frac{X - \mu}{\sigma},\]

which says how many standard deviations \(X\) is above (positive) or below (negative) its mean. By the rules for \(aX + b\) from level 7, \(Z\) has mean \(\tfrac{\mu - \mu}{\sigma} = 0\) and standard deviation \(\tfrac{\sigma}{\sigma} = 1\); and a shifted, stretched normal is still normal. So

\[P(X \le x) = P\!\left(Z \le \frac{x - \mu}{\sigma}\right).\]

Finding areas: \(\Phi(z)\)

The cumulative probability \(P(Z \le z)\) is written \(\Phi(z)\) (capital Greek "phi"). There is no simple formula for it, so it comes from a table, a calculator (normalcdf on a TI-84), a spreadsheet (NORM.S.DIST(z, TRUE)) or Python (statistics.NormalDist().cdf(z)). Some values:

\(z\) 0 0.5 1 1.25 1.5 1.645 1.96 2 2.326 2.5 3
\(\Phi(z)\) 0.5000 0.6915 0.8413 0.8944 0.9332 0.9500 0.9750 0.9772 0.9900 0.9938 0.9987

For example, \(P(Z < 1.5) = \Phi(1.5) \approx 0.9332\):

−3 −2 −1 0 1 2 3 z = 1.5 area 0.9332 z

Tables usually give only positive \(z\). Symmetry does the rest:

  • above a value: \(P(Z > z) = 1 - \Phi(z)\);
  • negative \(z\): \(\Phi(-z) = 1 - \Phi(z)\) (the area left of \(-z\) equals the area right of \(z\));
  • between two values: \(P(a < Z < b) = \Phi(b) - \Phi(a)\).

Always draw the curve and shade the area first. It takes ten seconds and catches most mistakes: if the shaded area is clearly less than half, your answer must be less than 0.5.

Working backwards

Sometimes you know the probability and want the value. Find \(z\) with \(\Phi(z)\) equal to the probability (the inverse normal, invNorm on a TI-84 or NORM.S.INV in a spreadsheet), then undo the z-score: \(x = \mu + z\sigma\). Useful values: \(\Phi(1.2816) = 0.90\), \(\Phi(1.645) = 0.95\), \(\Phi(1.96) = 0.975\), \(\Phi(2.326) = 0.99\).

Sums of normals are normal

If \(X\) and \(Y\) are independent and normal, then \(X + Y\) is also normal, with mean \(\mu_X + \mu_Y\) and variance \(\sigma_X^2 + \sigma_Y^2\) (the rules from level 7). This is special: the sum of two uniforms is a triangle, not a rectangle, but the sum of two bells is a bell. Lesson 5 shows that sums of many random parts of any shape end up close to a bell, which is why the normal is everywhere.

Is it really normal?

The normal is a model. Many measurements are roughly normal near the middle, but real data often have heavier tails, skew, or limits (a weight cannot be negative, yet every normal curve gives negative values a tiny probability). The model is excellent for "how likely is something within 2 or 3 standard deviations?" and much less reliable for "how likely is something 6 standard deviations away?". Checking whether data look normal is part of Statistics.

Worked examples

1. Underweight cereal. \(X \sim \operatorname{N}(505, 3^2)\). \(P(X < 500)\):

  • \(z = \tfrac{500 - 505}{3} = -1.67\) (to two decimal places).
  • \(P(Z < -1.67) = 1 - \Phi(1.67) \approx 1 - 0.9525 = 0.0475\). (With the unrounded \(z = -\tfrac53\), a calculator gives 0.0478.)

About 4.8% of boxes, roughly 1 in 21, are underweight.

2. Setting the machine. Keep \(\sigma = 3\) g. Choose \(\mu\) so that \(P(X < 500) = 0.01\). The z-score of 500 must be \(-2.326\) (because \(\Phi(-2.326) = 0.01\)):

\[\frac{500 - \mu}{3} = -2.326 \quad \Longrightarrow \quad \mu = 500 + 2.326 \times 3 \approx 507.0 \text{ g}.\]

Setting the average 2 g higher cuts underweight boxes from about 1 in 21 to 1 in 100. (A better machine with smaller \(\sigma\) would allow a lower setting and save cereal.)

3. Battery life. A battery's life is \(\operatorname{N}(500, 40^2)\) hours.

  • Chance it lasts less than 450 hours: \(z = \tfrac{450 - 500}{40} = -1.25\), \(P = 1 - 0.8944 = 0.1056\).
  • Chance it lasts between 480 and 560 hours: \(z\) from \(-0.5\) to \(1.5\), so \(P = \Phi(1.5) - \Phi(-0.5) = 0.9332 - (1 - 0.6915) = 0.9332 - 0.3085 = 0.6247\).
  • The life exceeded by only the best 10%: \(z = 1.2816\), so \(x = 500 + 1.2816 \times 40 \approx 551\) hours.

4. Two boxes. A shop sells cereal boxes in packs of two. Each box is \(\operatorname{N}(505, 3^2)\), independently. The pack total \(T\) is normal with mean 1010 and variance \(9 + 9 = 18\), so \(\sigma_T = \sqrt{18} \approx 4.24\). The chance the pack is under 1000 g: \(z = \tfrac{1000 - 1010}{4.24} \approx -2.36\), \(P \approx 0.009\). Under 1% for a pack, though almost 5% for a single box: the excesses and shortfalls of two boxes partly cancel.

Try it

  1. Find \(P(Z > 2)\), \(P(Z < -0.75)\) and \(P(-1 < Z < 2)\). (\(\Phi(0.75) = 0.7734\).)
  2. Find the \(z\) that has 95% of the area to its left, and the \(z\) that cuts off the top 2.5%.
  3. Test scores are \(\operatorname{N}(100, 15^2)\). What fraction score above 130? Between 85 and 115?
  4. In question 3, what score marks the top 10%?
  5. Use the 68-95-99.7 rule: heights in a group are roughly \(\operatorname{N}(170, 8^2)\) cm. About what fraction are taller than 186 cm? Between 162 and 178 cm?
  6. A bolt's length is \(\operatorname{N}(50, 0.2^2)\) mm. Bolts outside 49.6 to 50.4 mm are rejected. What fraction are rejected? If the machine drifts so the mean becomes 50.1 mm, what fraction are rejected now?
  7. Show that the quartiles of \(\operatorname{N}(\mu, \sigma^2)\) are about \(\mu \pm 0.674\sigma\), given \(\Phi(0.674) \approx 0.75\).
  8. \(X \sim \operatorname{N}(20, 3^2)\) and \(Y \sim \operatorname{N}(15, 4^2)\) are independent. Find the distribution of \(X - Y\), and \(P(X > Y)\).
Answers
  1. \(P(Z > 2) = 1 - 0.9772 = 0.0228\). \(P(Z < -0.75) = 1 - 0.7734 = 0.2266\). \(P(-1 < Z < 2) = 0.9772 - (1 - 0.8413) = 0.9772 - 0.1587 = 0.8185\).
  2. \(z = 1.645\). The top 2.5% starts at \(z = 1.96\).
  3. Above 130: \(z = 2\), \(P = 0.0228\), about 2.3%. Between 85 and 115: \(z = \pm 1\), about 68% (0.6827).
  4. \(100 + 1.2816 \times 15 \approx 119.2\).
  5. 186 is \(\mu + 2\sigma\). About 95% are within 2 standard deviations, so 5% outside, half of it above: about 2.5%. 162 to 178 is \(\mu \pm \sigma\): about 68%.
  6. The limits are \(z = \pm 2\), so \(1 - 0.9545 = 0.0455\), about 4.6% rejected. With mean 50.1: the limits become \(z = \tfrac{49.6 - 50.1}{0.2} = -2.5\) and \(z = \tfrac{50.4 - 50.1}{0.2} = 1.5\). Accepted: \(\Phi(1.5) - \Phi(-2.5) = 0.9332 - 0.0062 = 0.9270\), so about 7.3% rejected. A small drift raises the reject rate by more than half.
  7. The upper quartile \(q\) has \(P(X \le q) = 0.75\), so its z-score is 0.674 and \(q = \mu + 0.674\sigma\). By symmetry the lower quartile is \(\mu - 0.674\sigma\).
  8. \(X - Y\) is normal with mean \(20 - 15 = 5\) and variance \(9 + 16 = 25\) (variances add, even for a difference), so \(\sigma = 5\). \(P(X > Y) = P(X - Y > 0) = P(Z > -1) = 0.8413\).

Common mistake

Forgetting to standardise, or standardising with the variance. \(z = \tfrac{x - \mu}{\sigma}\) divides by the standard deviation. If a question says \(\operatorname{N}(500, 1600)\), then \(\sigma = 40\), not 1600.

Getting the side wrong. Tables give the area to the left. For "more than", subtract from 1. A quick sketch with the shaded area shows at once whether the answer should be above or below a half.

Where it leads