Surveys, experiments and observational studies
Three kinds of study
- A survey (or poll) asks people questions to estimate something about a population.
- An observational study watches groups without intervening — it can find associations but not causes.
- An experiment deliberately applies a treatment to one group and compares it with a control group.
Why only experiments can show cause
- In an experiment, participants are randomly assigned to the treatment or control group. Randomisation balances out other factors, so a difference in outcomes can be attributed to the treatment.
- In an observational study, the groups may differ in other ways (confounding variables), so you cannot tell whether the treatment or some other factor caused the difference.
Non-sampling errors
- Even a well-sized sample can be spoiled by non-sampling errors:
- Leading or confusing questions that push an answer.
- Non-response (people who don't reply may differ from those who do).
- Self-selection and interviewer bias.
A study finds that people who drink more coffee tend to sleep less, and concludes "coffee causes poor sleep." The data came from a survey. Evaluate the conclusion.
Step 1 — Identify the study type
It is an observational survey — people were asked about their habits, not randomly assigned to drink coffee or not.
Step 2 — Explain why cause cannot be claimed
Because there was no random assignment, a confounding variable could explain the link. For example, people with stressful jobs may both drink more coffee and sleep less — so stress, not coffee, could be the real cause.
Step 3 — State what would be needed
Only a randomised experiment (randomly assigning people to different coffee amounts) could support a causal claim.
Test yourself
Practice by grade
One question each at Achieved, Merit and Excellence. Have a go, then compare with the model answer.
Which type of study can establish that a treatment causes an effect?
Explain why an observational study cannot usually establish cause and effect.
A report claims a new teaching app 'improves grades', citing that students who chose to use it scored higher. Critique this and describe a study design that could support the causal claim.