How many HH's do I need on a villain to trust HUD stats?

I was teaching the z-tests for population proportions and thought of this poker application: how accurate are our HUD stats on villain based on a certain number of HH's? As a caution to the HUD-bots on FTR (I'm one), the accuracy is less than we probably think it is. That's why note-taking can be so valuable.

Without going too deeply into stats theory, a 95% confidence interval for a proportion stat like VP$P is constructed by computing a "Margin of Error" or ME% for the estimate our HUD gives us. So...

ME = 1.96 * sqrt( ( VP$P - ( 1 - VP$P ) ) / n )

Suppose villain's VP$P is 20% over 100 hands. Then

ME = 1.96 * sqrt( .2 * .8 / 100) = .078

So based on 100 HH's showing VP$P = 20%, we're estimating the actual VP$P is between 12.2 and 27.8. That's not quite accurate according to statistical theory, but it will do for our purposes.

The ME changes for different percentages and (generally) gets better as #HH's increases. Here are some examples.

VP$P = 10
ME for 50 hands = .083
ME for 100 hands = .059
ME for 500 hands = .026
ME for1k hands = .019

VP$P = 20
ME for 50 hands = .111
ME for 100 hands = .078
ME for 500 hands = .035
ME for1k hands = .025

VP$P = 40
ME for 50 hands = .136
ME for 100 hands = .096
ME for 500 hands = .043
ME for1k hands = .030

VP$P = 50
ME for 50 hands = .139
ME for 100 hands = .098
ME for 500 hands = .044
ME for1k hands = .030

I included values down to 10% because the ME's are the same for PFR. Summarizing, with about 50 HH's there's a plus/minus 8-15% ME on VP$P or PFR. For 500 HH's, there's a plus/minus 2-4% ME. And the ME increases for values close to 50% and decreases for those closer to 0% or 100%

I'm goofing off at work. When I get home tonight, I'll post some thoughts about statistical accuracy rates for stats like 3betting which need more HH's for the same accuracy since those decision points don't occur on every hand.

A couple of months ago I read a couple of threads on 2p2 about Bayes theorem and it's application in something that sounds similar to this and the conclusions people came up with were oddly dissimilar. Since you're clever enough to actually figure out the maths and it'd be a bit of a struggle for me I'll just throw out some of the ideas I remember from it.

If you sit down with a player you have never seen before and he doesn't put any money in once in ten hands you already have a strong indication that he's unlikely to be a 30+ vp$p player. Similarly, if he hasn't raised with PFR. Similarly, if he plays 8 out of the first 10 hands he's unlikely to have vp$p under 20%.

We can never achieve 95% confidence in any stats as no player plays completely mechanically - most are not good enough to do that, and those who are will be good enough to adjust. All we need is a ballpark figure. High or low is enough to get us started in terms of extracting value from stats.

It's very true that 20 hands doesn't tell you for sure if someone is 14/11 or 16/12 but it can point you in the right direction. Whatever information you have to work with is the information you have to work with - and while it's good to assess its quality you're often better off working on slightly inaccurate assumptions than completely disregarding what little you do know.

I guess what I'm saying is - what we'd want to judge based on our stats is not if they are completely accurate but rather how likely they are to be completely and utterly wrongheaded and misleading.

This said I COMPLETELY agree that reads are better than stats. Any hand shown down by any player is a valuable source of information as it will tell you exactly the decisions this player made with this pocket holding every street along the way. Treasure trove.

There is someone I play with quite a bit. Quite often I give him position. Hence, I think he tends to play a lower VP$IP/PFR when I'm at the table because I make it uncomfortable for him to enter pots light.

Also, post-flop reads are far easier to make against loose players than tight players for obvious reasons.

Anyway, you don't need to be scientific about it. Just look at the sample size, adjust a little for looseness and take the data for what it's worth along with the dozen other little pieces of information that feed into a good poker decision.

Originally Posted by Fnord

Anyway, you don't need to be scientific about it. Just look at the sample size, adjust a little for looseness and take the data for what it's worth along with the dozen other little pieces of information that feed into a good poker decision.

True - you don't have to be scientific, but I think it's a good caution for HUD-bots (I tend that way) not to get too wrapped up in stats and forget that other reads are vital, too, especially when we have < 200 hands against villain.

BTW, fnord, I used to take very few notes and posted one months ago. You abused me for it's inanity. You were right, and I've thought/worked on note taking a good bit since. My notes are actually helpful now, and consequently I take a lot more of them. Thanks.

now we just need for him to abuse you for tending towards being a HUD-bot. Pretty obviously not a 'skill' you want to be honing.

what you should emphasize is that when someone VPIPs 80/6 over 20 hands... he's not a TAG that just got dealt 16 pocket pairs in a row

Btw Robb - if you haven't read this you'll enjoy it:
http://archives1.twoplustwo.com/show...0#Post10933141