Topics

Bootstrap confidence intervals and making the call

Covers AS91582

The problem bootstrapping solves

You have one sample from each population, and one estimate of the difference between their medians.
You need to know how much that estimate would vary if you could take many samples — but you cannot go back and take more.
Bootstrapping simulates taking more samples, using only the data you already have.

How it works

The procedure, which the software carries out:
- Resample with replacement from your first sample, taking the same number of observations. Some values appear twice, some not at all.
- Do the same for your second sample.
- Calculate the difference in medians for that pair of resamples.
- Repeat — typically 1 000 times or more.
- The 1 000 differences form the bootstrap distribution.

Why resampling with replacement makes sense: your sample is the best available picture of the population, so drawing repeatedly from it mimics drawing repeatedly from the population. The spread of the bootstrap distribution estimates how much your difference-of-medians would vary between samples.

The confidence interval

The bootstrap confidence interval is the middle 95% of the bootstrap distribution — cut off the lowest 2.5% and the highest 2.5%.
It is reported as an interval for the difference between the population medians:

$\text{a plausible range of values for } (\text{median}_A - \text{median}_B)$

Always state which way round the subtraction goes. An interval of $(0.3,\ 1.5)$ means something different depending on which group was subtracted from which, and reports lose marks here routinely.

Making the call

This is the formal inference, and the rule is simple:

The interval	The call
Does not contain zero	A call can be made — the population medians differ, in the direction the interval shows
Contains zero	No call can be made — a difference of zero is a plausible value

Why zero is the dividing line: a difference of zero means the two population medians are the same. If zero is a plausible value for the difference, you cannot rule out that the populations are identical on this variable.

Writing the call

When a call can be made:
- "The bootstrap confidence interval for the difference in median sleep (Year 11 − Year 13) is 0.42 to 1.31 hours. Because this interval does not contain zero, I can make the call that back in the populations, Year 11 students tend to sleep longer than Year 13 students — by somewhere between about 25 minutes and 1 hour 19 minutes."
When no call can be made:
- "The interval is −0.15 to 0.87 hours. Because this interval contains zero, I cannot make a call: the evidence is not strong enough to say which population median is larger, or whether they differ at all."
Two things to avoid:
- Never say "there is no difference" when the interval contains zero. Say you cannot make a call. Absence of evidence is not evidence of absence, and this distinction is examined.
- Never call it "proof." A confidence interval is evidence about the populations, not certainty.

Interpreting the width of the interval

A narrow interval means the estimate is precise — you can say not just that there is a difference, but roughly how large.
A wide interval means the estimate is imprecise, and even a call may be of limited practical use.
What makes the interval narrower:
- larger samples
- less variable populations
Commenting on the width, and on what would narrow it, is a genuine reflection worth including.

Statistical against practical significance

An interval that excludes zero says a difference exists. It does not say the difference matters.
With very large samples, a tiny difference can produce an interval that excludes zero.
Always ask whether the size of the difference is practically important in the context, and say so. This is exactly the integration of statistical and contextual knowledge the Excellence criterion describes.

Tips and Tricks

State the direction of the subtraction before you report the interval. Write "Year 11 − Year 13" explicitly.

Say "I can make the call" or "I cannot make the call", and give the reason in terms of whether zero lies inside the interval. Markers look for that exact reasoning.

Never write "there is no difference". Write "I cannot make a call about which population median is larger."

Interpret the endpoints in context with units. "Between about 25 minutes and 1 hour 19 minutes more sleep" is far stronger than reporting two decimals.

Comment on whether the difference matters practically, not just whether the interval excludes zero. A difference can be real and trivial.

Say what would make the interval narrower — larger samples — and, if relevant, note that your samples were of a particular size.

Worked ExampleMaking a formal inference

Continuing the sleep investigation: the student bootstrapped 1 000 resamples of the difference in medians (Year 11 − Year 13) and obtained a 95% confidence interval of 0.35 to 1.42 hours.

Write the inference and the conclusion.

Step 1 — Report the interval, with its direction

Step 2 — Make the call

Step 3 — Interpret the endpoints in context

Step 4 — Comment on precision

Step 5 — Assess practical significance

Step 6 — Consider other relevant explanations

The inference establishes a difference between the year levels; it does not establish what causes it. Several explanations are plausible and the data cannot distinguish between them:

Assessment load. Year 13 students face NCEA Level 3 and university entrance requirements, with heavier workloads and more evening study.
Part-time employment. Year 13 students are more likely to hold jobs, often with evening or weekend shifts.
Social independence. Older students are more likely to have a driver's licence, later curfews and more evening commitments.
Biological changes in sleep timing. Adolescent circadian rhythms shift later with age, so Year 13 students may naturally fall asleep later while school start times stay fixed — meaning the difference is partly developmental rather than behavioural.

The fourth explanation is worth highlighting because it points a different way from the others: if the cause is largely biological, then advice to Year 13 students about time management would be addressing the wrong thing, and later school start times would be the relevant intervention.

A design limitation. This is a comparison of two different groups of students at one point in time, not the same students followed over two years. So it cannot separate the effect of being in Year 13 from any differences between these particular cohorts.

Step 7 — The conclusion

Answering the question. I investigated whether the median nightly sleep of Year 13 students tends to be lower than that of Year 11 students.

What the samples showed. In my samples of 40 students from each year level, the median sleep was 8.1 hours for Year 11 and 7.2 hours for Year 13, a difference of 0.9 hours. Both distributions were roughly symmetric with slight left skew and similar spreads, and they overlapped substantially.

The formal inference. The bootstrap 95% confidence interval for the difference in population medians (Year 11 − Year 13) was 0.35 to 1.42 hours. Because this does not contain zero, I can make the call that back in the populations, Year 11 students tend to sleep longer than Year 13 students, by somewhere between about 21 minutes and 1 hour 25 minutes per night.

Why this matters. The Year 13 median of 7.2 hours falls below the recommended 8 to 10 hours for adolescents, so this is a practically as well as statistically meaningful difference.

Limitations and reflection. The interval is wide, so while a difference is established its size is not precisely estimated; larger samples would narrow it. The study is observational and cross-sectional, so it identifies a difference between year levels without establishing a cause, and several plausible explanations exist — including a developmental shift in sleep timing that would have quite different implications from the workload explanation. Collecting data on bedtime, wake time, part-time work hours and evening screen use would allow those explanations to be distinguished, and following the same students from Year 11 to Year 13 would be a stronger design still.

Why this reaches Excellence

Bootstrap confidence intervals and making the call

Covers AS91582

The problem bootstrapping solves

You have one sample from each population, and one estimate of the difference between their medians.
You need to know how much that estimate would vary if you could take many samples — but you cannot go back and take more.
Bootstrapping simulates taking more samples, using only the data you already have.

How it works

The procedure, which the software carries out:
- Resample with replacement from your first sample, taking the same number of observations. Some values appear twice, some not at all.
- Do the same for your second sample.
- Calculate the difference in medians for that pair of resamples.
- Repeat — typically 1 000 times or more.
- The 1 000 differences form the bootstrap distribution.

Why resampling with replacement makes sense: your sample is the best available picture of the population, so drawing repeatedly from it mimics drawing repeatedly from the population. The spread of the bootstrap distribution estimates how much your difference-of-medians would vary between samples.

The confidence interval

The bootstrap confidence interval is the middle 95% of the bootstrap distribution — cut off the lowest 2.5% and the highest 2.5%.
It is reported as an interval for the difference between the population medians:

$\text{a plausible range of values for } (\text{median}_A - \text{median}_B)$

Always state which way round the subtraction goes. An interval of $(0.3,\ 1.5)$ means something different depending on which group was subtracted from which, and reports lose marks here routinely.

Making the call

This is the formal inference, and the rule is simple:

The interval	The call
Does not contain zero	A call can be made — the population medians differ, in the direction the interval shows
Contains zero	No call can be made — a difference of zero is a plausible value

Why zero is the dividing line: a difference of zero means the two population medians are the same. If zero is a plausible value for the difference, you cannot rule out that the populations are identical on this variable.

Writing the call

When a call can be made:
- "The bootstrap confidence interval for the difference in median sleep (Year 11 − Year 13) is 0.42 to 1.31 hours. Because this interval does not contain zero, I can make the call that back in the populations, Year 11 students tend to sleep longer than Year 13 students — by somewhere between about 25 minutes and 1 hour 19 minutes."
When no call can be made:
- "The interval is −0.15 to 0.87 hours. Because this interval contains zero, I cannot make a call: the evidence is not strong enough to say which population median is larger, or whether they differ at all."
Two things to avoid:
- Never say "there is no difference" when the interval contains zero. Say you cannot make a call. Absence of evidence is not evidence of absence, and this distinction is examined.
- Never call it "proof." A confidence interval is evidence about the populations, not certainty.

Interpreting the width of the interval

A narrow interval means the estimate is precise — you can say not just that there is a difference, but roughly how large.
A wide interval means the estimate is imprecise, and even a call may be of limited practical use.
What makes the interval narrower:
- larger samples
- less variable populations
Commenting on the width, and on what would narrow it, is a genuine reflection worth including.

Statistical against practical significance

An interval that excludes zero says a difference exists. It does not say the difference matters.
With very large samples, a tiny difference can produce an interval that excludes zero.
Always ask whether the size of the difference is practically important in the context, and say so. This is exactly the integration of statistical and contextual knowledge the Excellence criterion describes.

Tips and Tricks

State the direction of the subtraction before you report the interval. Write "Year 11 − Year 13" explicitly.

Say "I can make the call" or "I cannot make the call", and give the reason in terms of whether zero lies inside the interval. Markers look for that exact reasoning.

Never write "there is no difference". Write "I cannot make a call about which population median is larger."

Interpret the endpoints in context with units. "Between about 25 minutes and 1 hour 19 minutes more sleep" is far stronger than reporting two decimals.

Comment on whether the difference matters practically, not just whether the interval excludes zero. A difference can be real and trivial.

Say what would make the interval narrower — larger samples — and, if relevant, note that your samples were of a particular size.

Worked ExampleMaking a formal inference

Continuing the sleep investigation: the student bootstrapped 1 000 resamples of the difference in medians (Year 11 − Year 13) and obtained a 95% confidence interval of 0.35 to 1.42 hours.

Write the inference and the conclusion.

Step 1 — Report the interval, with its direction

Step 2 — Make the call

Step 3 — Interpret the endpoints in context

Step 4 — Comment on precision

Step 5 — Assess practical significance

Step 6 — Consider other relevant explanations

The inference establishes a difference between the year levels; it does not establish what causes it. Several explanations are plausible and the data cannot distinguish between them:

Assessment load. Year 13 students face NCEA Level 3 and university entrance requirements, with heavier workloads and more evening study.
Part-time employment. Year 13 students are more likely to hold jobs, often with evening or weekend shifts.
Social independence. Older students are more likely to have a driver's licence, later curfews and more evening commitments.
Biological changes in sleep timing. Adolescent circadian rhythms shift later with age, so Year 13 students may naturally fall asleep later while school start times stay fixed — meaning the difference is partly developmental rather than behavioural.

Step 7 — The conclusion

Answering the question. I investigated whether the median nightly sleep of Year 13 students tends to be lower than that of Year 11 students.

Why this matters. The Year 13 median of 7.2 hours falls below the recommended 8 to 10 hours for adolescents, so this is a practically as well as statistically meaningful difference.