1 / 21

Poker Bots 2008

Jeff Berkowitz pdxjjb@gmail.com. Poker Bots 2008. Tonight’s Talk. Why is poker an interesting problem? A bit about poker strategy A simple tournament bot A more interesting tournament bot Onward and beerward. Before we begin … a word about “tells”.

wallerj
Télécharger la présentation

Poker Bots 2008

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Jeff Berkowitz pdxjjb@gmail.com Poker Bots 2008

  2. Tonight’s Talk • Why is poker an interesting problem? • A bit about poker strategy • A simple tournament bot • A more interesting tournament bot • Onward and beerward

  3. Before we begin … a word about “tells” • Somebody always asks about Poker “tells” • “Computers can’t read tells!” (etc.) • Answer: tells are critical to success • … if you’re making a movie. • Seriously: tells are not a big part of poker. • Questions about tells usually come from people who haven’t yet grasped what a deep intellectual challenge the game is.

  4. Characterizing Games Nondeterministic Backgammon … Poker Stock market … Open Information Limited Information Chess Checkers Go … Battleship Minesweeper … Deterministic

  5. Strategy (1) – Equity Basics • You hold: 5 4 • Board shows: J T 6 2 • You believe your opponent has JJ or TT • She bets. Should you call? • It depends on the pot odds • 4 of the 46 unseen cards give you a likely winner • Pot odds must be 11 : 1 (or so) to call Limit Poker

  6. Strategy (2) – Controlling Equity • You Hold: K T • Board shows: K 7 5 • Pot contains 8 bets [note: small bets] • You bet your pair of kings [9 bets in pot] • Turns out, player on your left holds 9 8 • Your bet helped give him the odds to call • You should have checked, hoping to raise • Faced your opponent with calling 2 bets Limit Poker

  7. Deception (1) – Face Up Poker • Imagine Hold’em with hold cards face-up • We’ll say heads-up (2 players) for simplicity • All equity decisions are certain • Your turn: simulate a few million hands • < 1 second on modern CPUs • Model opponents as equally perfect • Bet based on the simulation outcome Limit Poker

  8. Deception (2) – Your Hidden Cards • Now imagine you get to hide hole cards • But the other player still plays face-up ;-) • You play the same “optimal” strategy • And your opponent knows it (important) • Your bet  positive expectation • Negative for them – so they will always fold • So you must “sandbag” some of your hands • Now defining “optimal” is harder Limit Poker

  9. NL Tourney Strategy (1) - Background • In general, for both limit and no-limit … • Pre-flop play is simpler, more formulaic • Because there is less information to go on • Post-flop play is harder • More information available • More time (flop, turn, river) • More room for deception, application of skill, …

  10. NL Tourney Strategy (2) • But in no-limit, post-flop play is optional • All-in on the pre-flop  no post-flop decisions! • Creates worst possible equity position for others • Only possible in no-limit • Especially important [later] in tournaments • “Advance of the blinds” • Consequence: “Move-in” or “All-in” players • Book “Kill Phil” [Rodman & Nelson 2005]

  11. Digression: Poker in Academia • University of Alberta Poker Research Group • Their work started long before the fad – 1995 • Numerous papers on the internet • www.poker.cs.ualberta.ca • Excellent software (100% pure Java) • Spinout company – Poker Academy • The company is actually called “Biotools” • Product has a plug-in API for bots, “Meerkat”

  12. Demo - KillPhilBot • Implements “Kill Phil Beginner” strategy • From ibid (“Kill Phil”, Rodman & Nelson) • Modified with a slightly smarter postflop

  13. Rules, rules, lotsa rules • A nontrivial rule-based bot gets hairy • If this else if this then that else if the other thing… • Tracing scattered throughout • Must restart Poker Academy to change the rules • Solution: separate behaviors from code • “Rule” (not really) interpretation system • “In the future, every programmer will have their own language” (me, 2008) • Poker Rules Definition Language - PRDL

  14. PRDLBot (“Phil Purdle”) • Express decision tree in PRDL • Parse with JJTree / javacc • Evaluate the parse tree for the action • No code generation (although there could be) • Advantages: • Put tracing in the engine not the rules definition • Change and reload without restarting PA • Not fully implemented yet

  15. PRDL Example (1) stage = PREFLOP // all identifiers refer back to bot core { m > 30 // “when we have a huge stack …” { holeCards in [KPGROUP_1, QQ, KQs] // use of a list of expressions { toCall <= BLIND // or write "numberOfRaises = 0" { bettors = 0 // no limpers { action [BET, (bankroll / 6)] // another use of a list of expressions } : // “otherwise, there are limpers”; push all-in { action [BET, bankroll] } } // CONTINUES NEXT SLIDE

  16. PRDL Example (2) : // otherwise, raised pot { numberOfRaises = 1 { action [BET, max[5 * raise, bankroll / 6]] } : // multiple raisers { action [BET, bankroll] } } } : // otherwise big stack, we don't have ultra-strong cards: we're done { action CHECK_FOLD // use of an expression that is not a list } } } : // Post-flop { // . . . . . etc . . . . . }

  17. PRDLBot Tour

  18. Bibliography (1 of 2) • U of Alberta (CA) Poker Research Group: • http://poker.cs.ualberta.ca/ • Scroll to Publications. Recommended there: • http://poker.cs.ualberta.ca/papers/AIJ02.html • Great online bot programming articles: • http://www.codingthewheel.com/category/poker • Poker Academy software: • http://www.poker-academy.com/ • http://forums.poker-academy.com/viewforum.php?f=3

  19. Bibliography (2 of 2) • Best overall poker web site / book publishers: • http://www.twoplustwo.com/ • Recommended books from them: • See http://twoplustwo.com/books.php • Best for beginners and for learning basic strategy are Ed Miller’s books, Gettings Started in Hold’em and Small Stakes Hold’em: Winning Big with Expert Play. • Best for learning tournament play are the Harrington and Robertie book series, Harrington on Hold’em (volumes 1 through 3, to be studied in order).

  20. Online Poker • Early years (1990s – 2002) • IRC, academic, first commercial sites • Fad (2003 – 2006) • TV with exposed hole cards, Chris Moneymaker • Reality (2006 - ) • UIGEA, bot suspicions • No individual has ever been arrested or charged with any crime for playing poker online in the United States.

  21. Cheating at Online Poker • General approach • Write bot to Meerkat interfaces • Test and debug to high level of skill in PA • Emulate Meerkat on a commercial poker client • Consider Omaha • 4 hole cards, must play 2 • Like playing 6 Hold’em hands simultaneously • Often played for a split high/low pot (12 hands) • Computer advantage is maximized

More Related