Re-randomisation and the causal call
The "chance alone" idea
- Suppose the treatment actually does nothing. Then the two groups differ only because of which units the random split happened to place where.
- Re-randomisation copies that chance process: pool all the results together, then randomly re-split them into two groups of the same sizes, and record the difference in centres.
- Do this many times (with technology) and you build the re-randomisation distribution — all the differences the random split produces when the treatment has no effect. It is centred on 0.
Comparing the observed difference
- Now place your observed difference on that distribution.
- The tail proportion is the fraction of re-randomised differences that are at least as extreme as the one you observed.
- Far out in the tail (small tail proportion) → the observed difference is hard to explain by chance → strong evidence the treatment caused it.
- In the middle of the distribution → the random split alone easily produces such a difference → not convincing evidence of an effect.
Making the call
- Make the call (the treatment had an effect) when the observed difference sits beyond most of the re-randomised differences — clearly in the tail.
- Do not make the call when the observed difference is a common value in the distribution — it could easily be chance.
In the fertiliser experiment, the treatment median yield was 4 kg above the control's, so the observed difference is 4 kg. A re-randomisation of the pooled data gives differences that are almost all between and kg, and only about 2% are 4 kg or more. Should the grower make the call?
Step 1 — Locate the observed difference
An observed difference of 4 kg lies beyond almost all of the re-randomised differences (which cluster around 0, within ).
Step 2 — Read the tail
Only about 2% of the chance-alone differences are as extreme as 4 kg — the observed difference is far out in the tail.
Step 3 — Make the call
Writing the causal conclusion
- A full conclusion states:
- The call — the treatment did (or did not) have an effect, and its direction.
- The evidence — the observed difference and where it fell on the re-randomisation distribution (in the tail, or not).
- The justification — random allocation is what allows a causal claim.
"The observed difference of 4 kg is far out in the tail of the re-randomisation distribution (only ~2% as extreme), so the difference is unlikely to be due to chance. Because the plants were randomly allocated, I conclude the fertiliser caused the higher yield for these plants."
Honest limits
- The causal claim applies to these experimental units; extending it to all plants everywhere needs the units to also be a random sample of that wider population.
- "Make the call" is never a proof — it says the effect is very unlikely to be chance, not impossible.
- If you cannot make the call, more replication (more units) may reveal a real but smaller effect.
Test yourself
Practice by grade
One question each at Achieved, Merit and Excellence. Have a go, then compare with the model answer.
A re-randomisation distribution is centred on 0. What does a difference of 0 represent?
An observed difference sits in the middle of the re-randomisation distribution, with about 40% of re-randomised differences as extreme. What should you conclude?
An experiment (units randomly allocated) gives an observed difference of 6, far out in the tail of the re-randomisation distribution (about 1% as extreme). Write a full conclusion, and explain what you can and cannot claim.