www.flopturnriver.com

Poker Forum

Over 1,246,000 Posts!

Already Registered? Username: Password: Remember Forgot Password
Tweet

Double or Nothings (DON) and Statistical Power

LinkBack

Thread Tools

Display Modes

Nakamura

11-08-2009, 02:11 PM Post subject: Double or Nothings (DON) and Statistical Power

#1 (permalink)

4-of-a-Kind

Join Date: Mar 2008

Location: Grindin'

Posts: 1,515

Nakamura is an unknown quantity at this point

With the rain beating down and feeling a bit sick, I was a bit bored today when I found my stats book. I've had something on my mind for a while now; I was wondering how many double or nothing tournaments do you need to play to be certain you are running at a particular level? If your mate is running at 7.5% over 2000 tournaments and you are running at 0% there is, statistically speaking, a good chance there is no difference in your ability. To understand why I have to give you a course in STATS101.

In statistics we create something called a null hypothesis (Ho), which assumes there is no difference between the data sets. In this case the null hypothesis would be that there is no difference between the mean percentage of tournaments your mate wins compared to you. The alternative hypothesis supposes that there are differences in the statistical behaviour of these two datasets.

The job of the statistician is to attempt to reject the null hypothesis, thereby accepting the alternative hypothesis. How do we do this? We use statistical tests to achieve this, but if our sample size is too small compared to the magnitude of the difference, we may not be able to detect the true difference.

Coming back to the hypotheses, since data is sampled at random, there is always a risk of reaching an incorrect conclusion. We can do this in two ways,

Type I error - The hypothesis is correct, but the test rejects it.
Type II error - The hypothesis is incorrect, but the test accepts it.

The probability of rejecting a false hypothesis is called the 'power' of a test and, you guessed it, there is a method for calculating this imaginatively called the power test.

Now, we are interested in the differences in ROI but we should look at the ITM %, since this the driving factor behind ROI. Conveniently, for DONs the result of any particular game is binary; either you lost the game or you finished ITM. So we can consider the ITM % as a proportion.

I think it suffices to say that there is a particular kind of power test for comparing proportions, which is based on the approximations with the normal distribution (the standard bell-shaped distribution that most random data conforms to). I have run this particular code in R, a free stats program that anyone can learn to use!

Here is the code:

Code:

> power.prop.test(power=0.85,p1=0.52,p2=0.53)

     Two-sample comparison of proportions power calculation 

              n = 44778.2
             p1 = 0.52
             p2 = 0.53
      sig.level = 0.05
          power = 0.85
    alternative = two.sided

 NOTE: n is number in *each* group

Eezy, peezy, Lionel squeezy, huh?

I have set the desired power to 85% (although it's up to the designer of the experiment). We are using a 95% confidence interval, which is where the significance level of 0.05 comes in (don't worry too much about this - it's pretty industry standard). I have set the first ITM percentage at 52%, which is exactly 0% ROI and the second ITM percentage at 53% or 1.9% ROI.

The power test indicates that you would need have 44778 under your belt to be sure that an ROI of 0% is statistically different to an ROI of 1.9% (at the 95% confidence interval). Now we can run with those ITM percentage and ask how many tournaments would you need to have under your belt to be sure that an ROI of X is different to an ROI of 0%. Here is the result.

Code:

ITM %      ROI %	      Tournaments
52	        0.0	           NA
53	        1.9	        44778
54	        3.8	        11181
55	        5.8	         4962
56	        7.7	         2786
57	        9.6	         1780
58	       11.5	         1233
59	       13.5	         904
60	       15.4	         690

Looking at it graphically, it looks like this.

Figure 1: The number of tournaments a player needs to play to be 95% certain that his ROI of 0% is statistically different to X ROI.

Obviously, the further apart the ROI figures are the fewer tournament you need to be certain that the ROI figures are in fact different. Interestingly this translates into a near perfect negative exponential line.

These results can perhaps be interpreted in a more practical manner with a player with a 5.8% ROI asking, "Can I be certain that my true ROI isn't 9.6%?" Since the difference of ITM finishes is 2% between those two ROI figures, we can simply read off the chart the numbers of tournaments you would need to play between an ITM% of 52 and and 54. In this case, you would need to play 11181 to be 95% certain that an ROI of 5.8% is statistically different to an ROI of 9.6%.

Well I hope this isn't too dense for folks to understand. I just did it as an exercise for myself and I hope some people find it interesting.

Edits: Just some housework with spelling.

Play for FREE and practice your game at...

Join the FTR Poker Forum to disable these banners and start posting!

rong

11-08-2009, 09:54 PM

#2 (permalink)

Full House

Join Date: Nov 2008

Location: Beachside

Posts: 1,196

Intereseting stuff Nak. I often have ideas like this while at work and plan to do some math research for a post when I get home, but I never get round to doing it for one reason or another. A+ for effort.

Quote:

Originally Posted by Micro2Macro

hey guys, if you ever make a snap call on the river when your opponent raises you're fucking retarded.

Fucking. Retarded.

Nakamura

11-09-2009, 08:16 AM

#3 (permalink)

4-of-a-Kind

Join Date: Mar 2008

Location: Grindin'

Posts: 1,515

Thanks Dan.

Another thought came to mind, which is this stuff obviously also applies to HUSNGS, since the outcome of a game is also binary.

taipan168

11-09-2009, 10:46 AM

#4 (permalink)

Administrator
Administrator

Join Date: Jul 2005

Location: Sydney

Posts: 10,441

taipan168 is an unknown quantity at this point

Thanks Rob, interesting analysis. When I have a little more time and headspace I'll run through your analysis in detail, I'm sure there are insights to be gained there!

rong

11-09-2009, 11:14 AM

#5 (permalink)

Full House

Join Date: Nov 2008

Location: Beachside

Posts: 1,196

Quote:

Originally Posted by Nakamura

Thanks Dan.

Another thought came to mind, which is this stuff obviously also applies to HUSNGS, since the outcome of a game is also binary.

And 3-way (tripple-ups), which I think I mentioned earlier are seriously soft at low levels.

Quote:

Originally Posted by Micro2Macro

hey guys, if you ever make a snap call on the river when your opponent raises you're fucking retarded.

Fucking. Retarded.

Arjonius

11-09-2009, 05:46 PM

#6 (permalink)

Straight

Join Date: Jul 2005

Posts: 122

What's a decent ROI in DoNs anyway?

rong

11-10-2009, 07:01 AM

#7 (permalink)

Full House

Join Date: Nov 2008

Location: Beachside

Posts: 1,196

One good thing that can be taken from this, is that if you are only just losing money or mayber breaking even, or even if you have lost considerably but don't have a huge data set, please continue playing.

It's probably not that you are crap at poker, it is likely you have just hit some variance and you are in fact a winning player. So continue playing for another, say 5000 SnG's before you either change your play or quit poker for good.

(Is that considered doing my bit for the poker community? I need donks in SnGs to make up for my small yet potentially large losses in ring games)

Quote:

Originally Posted by Micro2Macro

hey guys, if you ever make a snap call on the river when your opponent raises you're fucking retarded.

Fucking. Retarded.

kevster

11-10-2009, 12:09 PM

#8 (permalink)

Full House

Join Date: May 2006

Location: Fold City

Posts: 758

wow. just wow.

This post pleases me greatly. Very useful.

- You're the reason why paradise lost

Nakamura

11-10-2009, 03:11 PM

#9 (permalink)

4-of-a-Kind

Join Date: Mar 2008

Location: Grindin'

Posts: 1,515

Thanks Kev.

I'm been bored while watching us clobber the hell out the Zimbabwe in the cricket, so I wondered what if we change the significance level. In other words, you lower the statistical significance level, which increases the chance of a Type II error (accepting the alternative hypothesis when it is actually incorrect). In our case, that would mean you accept that there is difference between two ROI's when, in fact, they are not different.

I played with significance levels of 80% and 70%, which, needless to say, are unusual in the stats realm. Here are the results displayed in a graphical layout (since I can't be bothered to align everything)

.
Figure 2. ROI versus number of tournaments at three different sigificance levels, 95%, 80%, 75.

It just so happens that if you want to be 80% certain that two ROIs are in fact different, you now only need a sample size 59.8% compared to that at the 95% sig. level. For example, when answering whether an ROI of 3.8% is different to 0%, you now only 6691 tournaments compared to 11181. However, the flip-side is that there is a greater chance you are concluding that two ROI's are different, when in fact they are not.

If you lower your standards even further, you can get away with less then half the number of tournaments you required at the 95% sig level, but now you are accepting a 30% risk of error.

I doubt this is the end of my thoughts, as I was thinking about the evolution of certainty between samples. For example, when am I am 10%, 20% certain etc., etc. But this is it for now.

Latest Poker News
KoRnholio	05-26-2012, 03:08 PM Australia Legalized Online Poker coming up in next 6 to 12 Months
According to an email sent out by Mark Bryan, a gaming analyst at Merrill Lynch, the Australian government plans to legalize online poker sometime in the next six to 12 months. This move will coincide ... more at Poker Blogs »

« Previous Thread | Next Thread »

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

Posting Rules
You may not post new threads You may not post replies You may not post attachments You may not edit your posts BB code is On Smilies are On [IMG] code is On HTML code is Off Trackbacks are On Pingbacks are On Refbacks are On Forum Rules

All times are GMT. The time now is 08:11 AM.

Play Texas Holdem Online, Online Texas Holdem Strategy, & Poker Forum

This is not a gambling website.

Poker Forum

Over 1,246,000 Posts!

Poker Forum

Tournament Poker

Double or Nothings (DON) and Statistical Power

Play for FREE and practice your game at...