Why 10k hands before stats mean anything? (long and technical)
There is no firm number but here's a thought on statistics: (Yeah I know. Well I'm warning you now that what I'm about to say is mostly useless and will not make you a better poker player, so feel free leave now if you don't want to hear my rationalization.)
When we talk about a VPIP/PFR (or any stat), we are making an estimate of that frequency. We don't know the EXACT value of what that stat is, since it's constantly being updated by new information. Since we aren't 100% sure, we have some uncertainty. Like the VPIP is 20% +/- 2%. Easy enough.
Mathematicians have provided us with a whole field of study devoted to statistics and estimating them and figuring out how much confidence we can have in them.
Confidence Interval (CI): The range of values around the average which we are some % certain that the "actual" value will be after much more data is collected. It is tough, but the language goes, "We are x% sure that the answer is y% +/- z%."
I will choose x% to be19 out of 20 or 95%, which means, "If I do what I just did 20 more times, I expect that only ONCE will a result fall outside my uncertainty."
Why not choose 100%, and have certainty? In order to do that, our error range is infinite... The only thing we know for certain is that it IS a number. Why not narrow our error range down to +/-0%? Then we have 0% confidence in our estimate; we have no measure of how right or wrong it might be. So we HAVE to make a trade off to get both a non-zero CI and a non-infinite error range. (Try not to be distracted by the Heisenberg Principle here. We're talking about statistics in general, not the position and velocity of an electron.)
Now, a 95% CI requires about 5 successes to yield a percent that is equal in uncertainty to its value (more for very small percentages). *note: This is true for frequencies less than 50%. For frequencies greater than 50%, the uncertainty after 5 successes is 1-average% (I hate to say trust me, but trust me on this one.)
That is: if you do something 100 times and get a "successful result" 5 of those times, then your estimate of the frequency of a success is 5% +/- 5%. If you do something 20 times and are successful 4 times, then your frequency of success is 25% +/- 25%. If you flip a coin 10 times and get heads 5 times, then you know ABSOLUTELY NOTHING about whether that is a fair coin (50% +/- 50% = 0% - 100%). OK, so we understand that there is some minimal sample size that we need to have meaningful results.
What happens to our error range when we get more than 5 results? I'm not getting into the specifics of that here. I'll just say that the more data we have, the smaller our range of error becomes; the more robust our results.
1st point) If you are tracking a stat like CR (check-raise), and your villain has only had 6 times where they checked and it wasn't checked around, the stat of 17% is TOTALLY MEANINGLESS. It means that one time in 6 the villain has check-raised. One time is NOT enough to build a frequency!! The error range to within 95% CI includes negative numbers!! That's clearly not acceptable, as you can not have a negative chance of success.
OK, so why 10k hands? Here goes:
There are 169 significantly different starting hands. @ FR, there are 9 positions. It is not unreasonable to think of Holdem as an opportunity to play 169*9 = 1,521 different "games", each one has a specific pocket (with unspecific suits) and a specific position. In order to evaluate our success, we need to have played each game at least 5 times. 1,521*5 = 7,605 total hands before we can START to build a picture of our overall profitability. Remember this is the minimum possible sample that doesn't have negative frequencies in our error ranges.
Now, it's reasonable to think that not every pocket was played from every position exactly 5 times in our sample, huh? So even this "minimum" threshold of data is far too limited.
So let's double the number of hands we play from each position, to give ourselves a more stable data set. 169*9*10 = 15,210
What about the 6-max tables? 169*6*10 = 10,141
Tada!! There's the ~10k hands figure!!!
3rd point) This is still a small sample set, as we'd really prefer to have 60+ "successes" for whatever frequency we're estimating. It is important to note that up until now we're assuming that our ability to estimate stats is based solely on our pre-flop holdings and position. A sample of 60+ per pocket per position starts to account for all the different flops.
FR: 169*9*60 = 91,260
6-max: 169*6*60 = 60,840
So after this many hands, you can see a reasonable picture of how you play each hand from each position on a variety of flops. *note: we still haven't taken # of villains in the hand into account.