Designing a fair experiment
The principles of a good design
- Random allocation — split the units into groups by chance (draw names, use random numbers). This balances confounders across the groups.
- A control group — a group that gets no treatment (or a fake one), giving a baseline to compare against.
- Replication — enough units in each group. A difference based on two plants is far weaker than one based on twenty.
- Keep everything else the same — a fair test changes only the treatment between the groups; water, light, timing and measurement are kept identical.
Reducing bias
- Blinding — where possible, the units (and the person measuring) should not know which group they are in, so expectations do not sway the result.
- A placebo (a fake treatment) keeps the control group's experience the same apart from the active ingredient.
- Standardise measurement — measure the response the same way for every unit.
Writing the plan
- State the conjecture — what you think the treatment will do.
- Name the units, the treatment (and control), and how you will allocate randomly.
- Name the response variable and how you will measure it.
- State how many units go in each group.
Design an experiment to test whether a "study playlist" improves memory-test scores for a class of 30 students.
Step 1 — Conjecture and variables
- Conjecture: listening to the playlist while revising improves memory-test scores.
- Explanatory variable: playlist while revising, or silence.
- Response variable: score out of 20 on the memory test.
Step 2 — Random allocation
Randomly allocate the 30 students to two groups of 15 — e.g. draw names from a hat, or number them and use random numbers. Group A revises with the playlist; Group B revises in silence.
Step 3 — Keep it fair
Both groups revise the same material, for the same time, in the same room conditions, and sit the same test, marked the same way. The only difference is the playlist.
Step 4 — Replication
15 students per group is enough to see a difference beyond one or two unusual results.
Test yourself
Practice by grade
One question each at Achieved, Merit and Excellence. Have a go, then compare with the model answer.
Name two features of a well-designed experiment and say briefly what each does.
In a plant experiment, the treatment plants were kept on a sunny windowsill and the control plants in a shaded corner. Explain what is wrong and how to fix it.
A student tests a new sports drink by giving it to the school's first XV and comparing their sprint times with the times of the chess club. Critique this design and redesign it fairly.