Select Page
Poker Forum
Over 1,291,000 Posts!
Poker ForumAll Other Poker/Live Poker

using partial scoring to compare bots

Results 1 to 10 of 10
  1. #1
    Eric's Avatar
    Join Date
    Dec 2003
    Posts
    3,458
    Location
    California, USA

    Default using partial scoring to compare bots

    What do you think about having GlassJoeBot1 play GlassJoeBot2 100 pineapple hands and just keeping track of fouls and royalties?


    We could then do the same thing for BaldBullBot1 vs BaldBullBot2 and newbot1 vs newbot2.


    I think this would give us a partial idea about how the bots compare.
  2. #2
    Interesting idea.

    How about going one step further? Every bot plays every other bot.
  3. #3
    MadMojoMonkey's Avatar
    Join Date
    Apr 2012
    Posts
    10,322
    Location
    St Louis, MO
    This is a common way of comparing 2 AI's.


    I'd say 100 games is a really small sample.

    You can always run another 100 and see if the results are similar.
    This is not conclusive or quantifiable as far as verifying the results.
    Reproduce-ability a good rule of thumb to go by, though.

    Also, track as many stats as you can on early runs.
    You never know what questions you may ask in the future.
    It all depends on how much it costs to create and store the data.
  4. #4
    Eric's Avatar
    Join Date
    Dec 2003
    Posts
    3,458
    Location
    California, USA
    Quote Originally Posted by RoXmoon View Post
    Interesting idea.

    How about going one step further? Every bot plays every other bot.
    This would be great in a perfect world. Unfortunately we can't have this new bot play the others right now but we can look at hand histories of the bot playing itself.
  5. #5
    Eric's Avatar
    Join Date
    Dec 2003
    Posts
    3,458
    Location
    California, USA
    Quote Originally Posted by MadMojoMonkey View Post
    This is a common way of comparing 2 AI's.


    I'd say 100 games is a really small sample.
    True, we'll do 200 and it is really like 400 since it is bot vs bot.
  6. #6
    Eric's Avatar
    Join Date
    Dec 2003
    Posts
    3,458
    Location
    California, USA
    We ran this and here are the preliminary results (we still need to spot check some things):


    Results for GlassJoeBot (bot 1)
    hands: 200


    Player 1
    fouls: 69 for -414
    QQ top royalties: 1 for 7
    KK top royalties: 1 for 8
    AA top royalties: 1 for 9
    straight bottom royalties: 6 for 12
    flush bottom royalties: 28 for 112
    full house bottom royalties: 11 for 66
    other royalties: 17 for 65
    total foul and royalty points: -135
    avg f and r points per hand: -0.68


    Player 2
    fouls: 88 for -528
    QQ top royalties: 3 for 21
    KK top royalties: 1 for 8
    AA top royalties: 0 for 0
    straight bottom royalties: 9 for 18
    flush bottom royalties: 16 for 64
    full house bottom royalties: 5 for 30
    other royalties: 5 for 23
    total foul and royalty points: -364
    avg f and r points per hand: -1.82




    Results for BaldBullBot (bot 2)
    hands: 200


    Player 1
    fouls: 55 for -330
    QQ top royalties: 7 for 49
    KK top royalties: 6 for 48
    AA top royalties: 3 for 27
    straight bottom royalties: 13 for 26
    flush bottom royalties: 29 for 116
    full house bottom royalties: 17 for 102
    other royalties: 34 for 164
    total foul and royalty points: 202
    avg f and r points per hand: 1.01


    Player 2
    fouls: 45 for -270
    QQ top royalties: 10 for 70
    KK top royalties: 6 for 48
    AA top royalties: 6 for 54
    straight bottom royalties: 16 for 32
    flush bottom royalties: 29 for 116
    full house bottom royalties: 21 for 126
    other royalties: 21 for 71
    total foul and royalty points: 247
    avg f and r points per hand: 1.24




    Results for newbot (bot 3)
    hands: 200


    Player 1
    fouls: 34 for -204
    QQ top royalties: 20 for 140
    KK top royalties: 13 for 104
    AA top royalties: 11 for 99
    straight bottom royalties: 14 for 28
    flush bottom royalties: 60 for 240
    full house bottom royalties: 33 for 198
    other royalties: 65 for 393
    total foul and royalty points: 998
    avg f and r points per hand: 4.99


    Player 2
    fouls: 38 for -228
    QQ top royalties: 13 for 91
    KK top royalties: 11 for 88
    AA top royalties: 11 for 99
    straight bottom royalties: 17 for 34
    flush bottom royalties: 48 for 192
    full house bottom royalties: 36 for 216
    other royalties: 50 for 208
    total foul and royalty points: 700
    avg f and r points per hand: 3.50
  7. #7
    Eric's Avatar
    Join Date
    Dec 2003
    Posts
    3,458
    Location
    California, USA
    We did a spot check on these results and they look good.
  8. #8
    MadMojoMonkey's Avatar
    Join Date
    Apr 2012
    Posts
    10,322
    Location
    St Louis, MO
    Yep, I'd guess that each bot has gotten stronger than the last across the board.
  9. #9
    Bullbot must be pretty bad to have more negative points fouling than positive royalty points.

    Pretty dramatic improvement as noted by MadMojo
  10. #10
    Eric's Avatar
    Join Date
    Dec 2003
    Posts
    3,458
    Location
    California, USA
    Quote Originally Posted by stilltilting View Post
    Bullbot must be pretty bad to have more negative points fouling than positive royalty points.
    I think you mean Glass Joe.

    Bald Bull scores more in royalties than he loses in fouls but he's not that great.

    I was hoping we were going to add a new bot who is better than Bald Bull but it would cost some serious money to do that and I'm not sure how we'd get that money back since we don't monetize this ofc game.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •