I was teaching the z-tests for population proportions and thought of this poker application: how accurate are our HUD stats on villain based on a certain number of HH's? As a caution to the HUD-bots on FTR (I'm one), the accuracy is less than we probably think it is. That's why note-taking can be so valuable.

Without going too deeply into stats theory, a 95% confidence interval for a proportion stat like VP$P is constructed by computing a "Margin of Error" or ME% for the estimate our HUD gives us. So...

ME = 1.96 * sqrt( ( VP$P - ( 1 - VP$P ) ) / n )

Suppose villain's VP$P is 20% over 100 hands. Then

ME = 1.96 * sqrt( .2 * .8 / 100) = .078

So based on 100 HH's showing VP$P = 20%, we're estimating the actual VP$P is between 12.2 and 27.8. That's not quite accurate according to statistical theory, but it will do for our purposes.

The ME changes for different percentages and (generally) gets better as #HH's increases. Here are some examples.

VP$P = 10
ME for 50 hands = .083
ME for 100 hands = .059
ME for 500 hands = .026
ME for1k hands = .019

VP$P = 20
ME for 50 hands = .111
ME for 100 hands = .078
ME for 500 hands = .035
ME for1k hands = .025

VP$P = 40
ME for 50 hands = .136
ME for 100 hands = .096
ME for 500 hands = .043
ME for1k hands = .030

VP$P = 50
ME for 50 hands = .139
ME for 100 hands = .098
ME for 500 hands = .044
ME for1k hands = .030

I included values down to 10% because the ME's are the same for PFR. Summarizing, with about 50 HH's there's a plus/minus 8-15% ME on VP$P or PFR. For 500 HH's, there's a plus/minus 2-4% ME. And the ME increases for values close to 50% and decreases for those closer to 0% or 100%

I'm goofing off at work. When I get home tonight, I'll post some thoughts about statistical accuracy rates for stats like 3betting which need more HH's for the same accuracy since those decision points don't occur on every hand.