Skip to content

Sampling distributions

Statistics and data · level 8, lesson 1 of 7

A question to start

The adult women of a town have heights with mean 163 cm and standard deviation 7 cm. About one woman in six is taller than 170 cm. Now pick 25 women at random and work out their mean height. How likely is it that this sample mean is above 170 cm?

Most people guess "about one in six again". The true answer is far smaller than one in a million. A single person can easily be tall; a whole random group of 25 averaging that tall almost never happens. Averages are much steadier than single values, and this lesson measures exactly how much steadier. Every poll, survey and experiment in this level rests on that one fact.

The idea

Parameters, statistics and sampling variability

A parameter is a number that describes a whole population: the true mean height \(\mu\) of every woman in the town, the true proportion \(p\) of voters who support a plan. Usually we cannot measure it directly.

A statistic is a number computed from a sample: the sample mean \(\bar{x}\), the sample proportion \(\hat{p}\) ("p-hat"). We use statistics to estimate parameters.

Different random samples give different statistics. That is sampling variability. It is not a mistake; it is what happens when you look at part of a population chosen by chance. The question is how big it is.

The sampling distribution

Imagine taking every possible random sample of size \(n\) from the population and computing the statistic for each. The distribution of all those values is the sampling distribution of the statistic. It answers: if I take one sample, what values of the statistic might I get, and how likely is each?

You can see one exactly with dice. One roll of a fair die is a population with values 1 to 6, equally likely: mean \(\mu = 3.5\) and standard deviation \(\sigma \approx 1.708\). Now take samples of 2 rolls and compute the mean of each sample. There are 36 equally likely pairs, and their means run from 1 (a pair of 1s) to 6 (a pair of 6s). Then do the same with samples of 5 rolls (\(6^5 = 7776\) possible samples).

one roll mean of 2 rolls mean of 5 rolls 1 6 1 6 1 6

Three things happen as the sample size \(n\) grows:

  1. Centre: every sampling distribution is centred on 3.5, the population mean.
  2. Spread: the spread shrinks. Means of 5 rolls rarely get far from 3.5.
  3. Shape: the shape becomes a bell, even though the population is flat.

Centre: the sample mean is unbiased

The mean of the sampling distribution of \(\bar{x}\) is exactly \(\mu\). A statistic whose sampling distribution is centred on the parameter is called unbiased: it does not systematically overshoot or undershoot. Any single sample mean misses \(\mu\) a little, but the misses balance out over all possible samples.

Bias and variability are different things. Think of shots at a target:

low bias, low variability low bias, high variability high bias, low variability high bias, high variability

Bias is aiming at the wrong place; it comes from how the sample is chosen (level 5) and a bigger sample does not fix it. Variability is scatter; a bigger random sample does reduce it.

Spread: the square-root law

The standard deviation of the sampling distribution of \(\bar{x}\) is

\[\sigma_{\bar{x}} = \frac{\sigma}{\sqrt{n}}.\]

It is often called the standard error of the mean. Here is why it is true.

The mean is the sum divided by \(n\): \(\bar{x} = \dfrac{X_1 + X_2 + \dots + X_n}{n}\), where the \(X\)s are the \(n\) randomly chosen values. For values chosen independently, a basic fact of probability says that variances add (the variance is the square of the standard deviation; see Probability, level 7: variance). So the sum has variance \(\sigma^2 + \sigma^2 + \dots + \sigma^2 = n\sigma^2\).

Dividing a quantity by \(n\) divides its standard deviation by \(n\), and so its variance by \(n^2\). So

\[\text{variance of } \bar{x} = \frac{n\sigma^2}{n^2} = \frac{\sigma^2}{n}, \qquad \text{standard deviation of } \bar{x} = \frac{\sigma}{\sqrt{n}}.\]

Why do variances add, rather than standard deviations? Because independent values partly cancel: a high roll is often paired with a low one. Adding standard deviations would assume they always push the same way.

Check with two dice: the 36 equally likely means have variance \(\dfrac{35}{24} \approx 1.458\), which is exactly half of a single die's variance \(\dfrac{35}{12} \approx 2.917\). Standard deviation \(\dfrac{1.708}{\sqrt{2}} \approx 1.208\). ✓

The square root is important and a little disappointing: to halve the standard error you need four times as many values; to cut it to a tenth you need a hundred times as many.

Shape: the central limit theorem

The central limit theorem says that for a large enough random sample, the sampling distribution of \(\bar{x}\) is close to normal, whatever the shape of the population. The more skewed the population, the larger \(n\) must be; a common rule of thumb is \(n \ge 30\), and for symmetric populations much smaller samples are fine. If the population itself is normal, \(\bar{x}\) is exactly normal for every \(n\).

So with three facts (centre \(\mu\), spread \(\sigma/\sqrt{n}\), shape normal) you know the whole sampling distribution, and you can use z-scores and the 68–95–99.7 rule from level 7.

Sample proportions

For a yes-or-no question, code "yes" as 1 and "no" as 0. Then the sample proportion \(\hat{p}\) is just the sample mean of the 0s and 1s, and the population of 0s and 1s has mean \(p\) and standard deviation \(\sqrt{p(1 - p)}\). So the square-root law gives

\[\text{standard deviation of } \hat{p} = \sqrt{\frac{p(1 - p)}{n}},\]

centred on \(p\), and close to normal when \(np\) and \(n(1 - p)\) are both at least about 10.

Worked examples

1. The tall sample, solved. Heights: \(\mu = 163\), \(\sigma = 7\). Samples of \(n = 25\).

  • Standard error: \(\dfrac{7}{\sqrt{25}} = \dfrac{7}{5} = 1.4\) cm.
  • About 95% of sample means lie within 2 standard errors of 163: between \(163 - 2.8 = 160.2\) and \(163 + 2.8 = 165.8\) cm.
  • A sample mean of 170 is \(z = \dfrac{170 - 163}{1.4} = 5\) standard errors above \(\mu\). The chance is about 3 in 10 million.

A single woman above 170 cm: \(z = 1\), about 16%. A sample mean above 170: almost never.

2. How big a sample? How large must \(n\) be for the standard error of the mean height to be 0.5 cm?

\(\dfrac{7}{\sqrt{n}} = 0.5\) gives \(\sqrt{n} = 14\), so \(n = 196\).

3. A poll. In a city, 40% of adults support a new bike lane (\(p = 0.4\)). A pollster asks a random sample of 100.

  • The sampling distribution of \(\hat{p}\) is centred on 0.40 with standard deviation \(\sqrt{\dfrac{0.4 \times 0.6}{100}} = \sqrt{0.0024} \approx 0.049\), about 4.9 percentage points.
  • About 95% of such polls give between \(40 - 9.8 \approx 30\%\) and \(40 + 9.8 \approx 50\%\). A single poll of 100 people could easily say 33% or 47%. With \(n = 1000\) the standard deviation drops to about 1.5 points.

4. A skewed population. Waiting times at a clinic are skewed right with \(\mu = 6\) minutes and \(\sigma = 5\) minutes. For samples of 50 patients, describe the sampling distribution of the mean wait.

Centre 6 minutes; standard error \(\dfrac{5}{\sqrt{50}} \approx 0.71\) minutes; shape close to normal by the central limit theorem (the sample is large), even though single waits are skewed. So about 95% of sample means fall between about 4.6 and 7.4 minutes.

Try it

  1. Say whether each number is a parameter or a statistic: (a) 52% of the 1000 people surveyed said yes; (b) the mean age of all Canadians recorded by the census; (c) the mean mass of 30 apples picked at random from a truck.
  2. A population has \(\sigma = 20\). Find the standard error of \(\bar{x}\) for \(n = 4\), \(n = 16\), \(n = 100\), \(n = 400\).
  3. Exam marks have \(\mu = 68\) and \(\sigma = 12\). For random samples of 36 students, between what two values do about 95% of sample means lie?
  4. Why does doubling the sample size not halve the standard error? What does it do?
  5. A population of 0s and 1s has \(p = 0.5\). Find the standard deviation of \(\hat{p}\) for \(n = 100\) and \(n = 2500\).
  6. A website asks visitors to vote, and 50 000 do. Does the square-root law tell you the result is within a fraction of a percent of the true view of all Canadians? Explain using bias and variability.
  7. In example 1, what is the chance that a sample of 25 women has mean height above 165 cm? (Use technology or the z-table.)
Answers
  1. (a) statistic (from a sample); (b) parameter (the whole population, if the census counted everyone); (c) statistic.
  2. \(20 \div 2 = 10\); \(20 \div 4 = 5\); \(20 \div 10 = 2\); \(20 \div 20 = 1\).
  3. Standard error \(12 \div 6 = 2\). About 95% lie within \(2 \times 2 = 4\) of 68: between 64 and 72.
  4. The standard error is \(\sigma / \sqrt{n}\). Doubling \(n\) multiplies \(\sqrt{n}\) by \(\sqrt{2} \approx 1.41\), so the standard error is divided by 1.41: it falls by about 29%, not 50%.
  5. \(\sqrt{0.25/100} = 0.05\) (5 points); \(\sqrt{0.25/2500} = 0.01\) (1 point).
  6. No. The square-root law only measures variability of random samples. A voluntary online vote is biased: the people who choose to vote are not like Canadians in general. A large biased sample gives a tight cluster of shots in the wrong place.
  7. \(z = \dfrac{165 - 163}{1.4} \approx 1.43\). About 92.3% are below, so about 7.7% are above.

Common mistake

Using \(\sigma\) instead of \(\sigma/\sqrt{n}\). Questions about one value use the population standard deviation. Questions about a sample mean use the standard error. Ask: is this about one person or about the average of a group?

Thinking a big sample cures bias. A bigger sample shrinks random scatter only. If the sampling method is unfair, a bigger sample is just more confidently wrong.

Where it leads