What a margin of error is and what it covers

The idea

• A sample gives an estimate, and a different sample would have given a different estimate. The margin of error says how far the estimate could plausibly be from the population value because of sampling variability alone.
• The standard convention, used by every published poll, is a 95% margin of error: in about 95 of every 100 samples, the interval "estimate ± margin of error" contains the true population value.

The approximation you are expected to use

• For a percentage estimated from a random sample of size $n$, the approximate 95% margin of error is:

$$\text{MoE} \approx \frac{1}{\sqrt{n}}$$

• Expressed as a percentage, multiply by 100.
  • $$n = 100 \Rightarrow \text{MoE} \approx 10\%$$
  • $$n = 400 \Rightarrow \text{MoE} \approx 5\%$$
  • $$n = 1000 \Rightarrow \text{MoE} \approx 3.2\%$$
  • $$n = 2500 \Rightarrow \text{MoE} \approx 2\%$$
  • $$n = 10\,000 \Rightarrow \text{MoE} \approx 1\%$$
• This is the maximum margin of error, and it applies exactly when the percentage is near 50%. For percentages far from 50% the true margin of error is smaller — which is why a party on 4% is measured more precisely than a party on 48%.

The square-root law

• Because $n$ appears under a square root, precision improves slowly:
  • To halve the margin of error you must quadruple the sample size.
  • Going from $n = 1000$ to $n = 2000$ improves the margin of error only from 3.2% to 2.2%.
• This is why polls stop at about 1 000. The cost doubles, and the precision improves by one percentage point. It is a deliberate economic choice, not carelessness — a useful point to make when a report is criticised for its sample size.
• Population size is irrelevant. A sample of 1 000 gives the same margin of error whether the population is 100 000 or 5 million. Students find this counter-intuitive; it is nonetheless true, and reports that say "only 1 000 people out of 5 million" are making a mistake you can name.

What the margin of error covers — and does not

Source of error	Covered by the margin of error?
Different random samples give different estimates	Yes
Undercoverage of part of the population	No
Non-response	No
Leading or ambiguous questions	No
Respondents not telling the truth	No
Data entry and processing mistakes	No

• Everything below the first line is non-sampling error, and the margin of error makes no allowance for any of it.
• This is the single most examined idea on this page. A poll's ±3.1% is an honest statement about one source of error and silence about five others.

Reading a margin of error in context

• If a report says "43%, margin of error 3.1%", the correct reading is:

We estimate 43%. Values between roughly 39.9% and 46.1% are consistent with what we observed. We are using a method that captures the true value about 95% of the time.

• Statements to avoid, because they are wrong:
  • "There is a 95% chance the true value is between 39.9% and 46.1%." The true value is fixed; it is the interval that varies from sample to sample.
  • "95% of people gave an answer in that range." The interval is about the estimate, not about individuals.
  • "The result is accurate to within 3.1%." Only the sampling part of the error is.

Margins of error for other quantities

• Reports also quote margins of error for means (average income, average waiting time). The $1/\sqrt{n}$ rule is for percentages only — a mean's margin of error depends on the spread of the data as well as $n$.
• If a report gives a margin of error for a mean, use the number it gives; do not try to recompute it with $1/\sqrt{n}$.