How many trials? Why more is better

The estimate varies

• A simulation gives an estimate, not the exact probability. Run it again and you get a slightly different number — this is chance variation.
• With only a few trials, the estimate can be well off. One unlucky run of 10 trials can give a wildly wrong figure.

More trials settle the estimate

• As the number of trials grows, the running estimate stops jumping around and settles toward the true probability. This is the long-run relative frequency.

• Early on (few trials) the estimate swings widely; by several hundred trials it is close to and stable near the true value.
• This is why a good simulation uses many trials — the more you run, the more you can trust the estimate.

Comparing with theory

• If the true probability can be calculated, the simulation estimate should be close to it — and closer the more trials you run.
• A large gap has three possible causes, and naming which one applies is what the higher grades are looking for:
  • too few trials — the estimate simply has not settled yet
  • a design error — the random numbers were assigned in the wrong proportion, or a trial was defined wrongly
  • the model does not match the situation — for example assuming each free throw is independent when a player's shots may not be
• Check the design before blaming chance. If a simulation of a fair coin settles near 0.7 after 500 trials, the problem is not luck; something in the assignment is wrong.
• Say how close is close enough. An estimate of 0.52 from 500 trials against a theoretical 0.5 is excellent agreement; a difference of 0.2 after 500 trials is not chance.

Writing the conclusion

• Answer the original question, in context and in words, not just as a number.
• State the estimate as an estimate, and say how many trials it came from: "based on 500 trials, the estimated probability is about 0.53".
• Comment on how reliable it is — more trials, more reliable — and compare with the theoretical probability if one exists.
• Name the assumptions your simulation made, and say whether they are reasonable. Every simulation assumes something: that the events are independent, that the probability stays constant, that nothing outside the model matters.
• This is where statistical insight is earned. "The simulation assumes each shot is independent with a fixed 70% chance, but a real player's chance may drop under pressure or rise once they find their rhythm, so the estimate may not reflect a real game" is the kind of reflection the standard is asking for.