Double or Nothings (DON) and Statistical Power

With the rain beating down and feeling a bit sick, I was a bit bored today when I found my stats book. I've had something on my mind for a while now; I was wondering how many double or nothing tournaments do you need to play to be certain you are running at a particular level? If your mate is running at 7.5% over 2000 tournaments and you are running at 0% there is, statistically speaking, a good chance there is no difference in your ability. To understand why I have to give you a course in STATS101.

In statistics we create something called a null hypothesis (Ho), which assumes there is no difference between the data sets. In this case the null hypothesis would be that there is no difference between the mean percentage of tournaments your mate wins compared to you. The alternative hypothesis supposes that there are differences in the statistical behaviour of these two datasets.

The job of the statistician is to attempt to reject the null hypothesis, thereby accepting the alternative hypothesis. How do we do this? We use statistical tests to achieve this, but if our sample size is too small compared to the magnitude of the difference, we may not be able to detect the true difference.

Coming back to the hypotheses, since data is sampled at random, there is always a risk of reaching an incorrect conclusion. We can do this in two ways,

Type I error - The hypothesis is correct, but the test rejects it.
Type II error - The hypothesis is incorrect, but the test accepts it.

The probability of rejecting a false hypothesis is called the 'power' of a test and, you guessed it, there is a method for calculating this imaginatively called the power test.

Now, we are interested in the differences in ROI but we should look at the ITM %, since this the driving factor behind ROI. Conveniently, for DONs the result of any particular game is binary; either you lost the game or you finished ITM. So we can consider the ITM % as a proportion.

I think it suffices to say that there is a particular kind of power test for comparing proportions, which is based on the approximations with the normal distribution (the standard bell-shaped distribution that most random data conforms to). I have run this particular code in R, a free stats program that anyone can learn to use!

Here is the code:

Intereseting stuff Nak. I often have ideas like this while at work and plan to do some math research for a post when I get home, but I never get round to doing it for one reason or another. A+ for effort.

Thanks Dan.

Another thought came to mind, which is this stuff obviously also applies to HUSNGS, since the outcome of a game is also binary.

Thanks Rob, interesting analysis. When I have a little more time and headspace I'll run through your analysis in detail, I'm sure there are insights to be gained there!

What's a decent ROI in DoNs anyway?

One good thing that can be taken from this, is that if you are only just losing money or mayber breaking even, or even if you have lost considerably but don't have a huge data set, please continue playing.

It's probably not that you are crap at poker, it is likely you have just hit some variance and you are in fact a winning player. So continue playing for another, say 5000 SnG's before you either change your play or quit poker for good.

(Is that considered doing my bit for the poker community? I need donks in SnGs to make up for my small yet potentially large losses in ring games)

wow. just wow.

This post pleases me greatly. Very useful.

Thanks Kev.

I'm been bored while watching us clobber the hell out the Zimbabwe in the cricket, so I wondered what if we change the significance level. In other words, you lower the statistical significance level, which increases the chance of a Type II error (accepting the alternative hypothesis when it is actually incorrect). In our case, that would mean you accept that there is difference between two ROI's when, in fact, they are not different.

I played with significance levels of 80% and 70%, which, needless to say, are unusual in the stats realm. Here are the results displayed in a graphical layout (since I can't be bothered to align everything)

.
Figure 2. ROI versus number of tournaments at three different sigificance levels, 95%, 80%, 75.

It just so happens that if you want to be 80% certain that two ROIs are in fact different, you now only need a sample size 59.8% compared to that at the 95% sig. level. For example, when answering whether an ROI of 3.8% is different to 0%, you now only 6691 tournaments compared to 11181. However, the flip-side is that there is a greater chance you are concluding that two ROI's are different, when in fact they are not.

If you lower your standards even further, you can get away with less then half the number of tournaments you required at the 95% sig level, but now you are accepting a 30% risk of error.

I doubt this is the end of my thoughts, as I was thinking about the evolution of certainty between samples. For example, when am I am 10%, 20% certain etc., etc. But this is it for now.

Poker Forum

Join Our Poker Community!

Poker Forum Index

Sit-n-Go Tourney Tactics

Double or Nothings (DON) and Statistical Power

Poker Forum Index

Sit-n-Go Tourney Tactics

Double or Nothings (DON) and Statistical Power