|
lol. BIG.
When you see the 66% (2/3), what you see is that there were 2 successes in 3 trials. So the MINIMUM uncertainty is +/- 33% (1 over the # of trials). That just represents the granularity of your estimate. I.e. with only 3 trials, you can NOT have 50% (only 0%, 33%, 67%, 100%). So if the "actual" percentage is 52%, well, you'll need 50 trials to see a granularity of 2%
That says nothing to the notion of convergence. tl;dr
Here's some examples to give you a feel for error on percentages:
E.g. The "actual" FT3 is 67%.
After 100 trials, assuming we have 67 folds, then we estimate the FT3 to be
67% +/- 10% @ 95% confidence.
After 1000 trials: 67% +/- 3% @ 95% confidence.
If the percentage you're estimating is very close to 0% or 100%, then things get tighter.
E.g. The "actual" 4-bet% is 4%
After 100 trials: 4% +/- 3.9% @ 95% confidence.
After 1000 trials: 4% +/- 1.2% @ 95% confidence
TL;DR
You need tens of trials to get a ballpark
You need hundreds of trials to nail down a narrow-ish range
You need thousands of trials to make a read that is within ~3% of the "actual" value.
|