Designing a fair experiment
From question to design
- Write a clear investigative question: does this treatment change this measured outcome?
- Identify the experimental units (who or what receives the treatment), the treatment(s), and the response variable (what you measure).
Allocating fairly
- List the units and use a random mechanism — a random number generator, or drawing numbered slips — to split them into groups.
- Use equal-sized groups where possible, each with a decent number of units (replication), so the comparison is reliable and not swayed by one or two individuals.
Controlling other variables
- Keep everything except the treatment identical for both groups — same time of day, same instructions, same measuring method and equipment.
- Anything that differs between the groups besides the treatment becomes a confounder and ruins the comparison.
- Use blinding where you can: if participants (and the people measuring) do not know which group they are in, expectation bias is removed.
Reducing bias
- Randomise the order in which units are measured.
- Use objective, consistent measurements.
- A placebo (a dummy treatment) keeps the control group's experience the same apart from the active ingredient, so the comparison isolates the treatment.
Design an experiment to test whether listening to music while studying affects how many words from a 20-word list a student can recall.
Step 1 — Question, units, treatment, response
Question: does studying with music change the number of words recalled? Units: the student volunteers. Treatment: studying the list with music; control: studying it in silence. Response: number of words correctly recalled out of 20.
Step 2 — Random allocation
Randomly allocate the volunteers into two equal groups using a random number generator — one group studies with music, the other in silence.
Step 3 — Control other variables
Give both groups the same list, the same study time (say 3 minutes), the same room, and the same recall test immediately afterward. Only the presence of music differs.
Step 4 — Measure and compare
Record each student's score, then compare the two groups' distributions of words recalled.
Test yourself
Practice by grade
One question each at Achieved, Merit and Excellence. Have a go, then compare with the model answer.
In an experiment testing whether a fertiliser increases tomato yield, plots are treated with fertiliser or left untreated and the mass of tomatoes is recorded. State the treatment and the response variable.
For the tomato experiment above, describe how you would randomly allocate the plots and list two variables you would control.
Design a fair experiment to test whether a sports drink improves 100 m sprint time compared with water, for a squad of 24 athletes. Justify the key design choices.