1 / 16

Neural Network Implementation of Poker AI

Neural Network Implementation of Poker AI. Christopher Kramer. Outline of Information. The Challenge Application, problem to be solved, motivation Why create a poker machine with ANNE? The Flop The hypothesis Can a Poker AI run using only an ANNE? The Turn The approach and methodology

anika
Télécharger la présentation

Neural Network Implementation of Poker AI

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Neural Network Implementation of Poker AI Christopher Kramer

  2. Outline of Information • The Challenge • Application, problem to be solved, motivation • Why create a poker machine with ANNE? • The Flop • The hypothesis • Can a Poker AI run using only an ANNE? • The Turn • The approach and methodology • How is this machine created? • The River • Experimentation • How should it be tested? • The Showdown • Conclusions and future work • Did it work?

  3. The Challenge • The approach focused on creating an AI capable of playing poker at a reasonable level of play using Neural Networks. • The difficulty: • Incomplete knowledge • Multiple competing agents • Risk management • Opponent modeling • Deception • Unreliable information • Creating solutions to address these difficulties can be applied to other problems with similar domains. NOISE!

  4. The Fold (Hypothesis) • The strengths of computers: • Quick computation • Large memory • Purely logical • Using a series of machine learning techniques including neural networks, it is predicted that a machine will be able to play at a moderate level while ignoring typical features of poker programs such as statistical analysis and pot odds.

  5. The Turn (Approach) • Problem split into a number of stages

  6. Pre-Flop • Hardcoded • Action chosen based off of known values of starting hands The Turn (continued) • Information gathered from percentage of games won with starting hand from poker databases • All hands in top 30th percentile were analyzed for characteristics which were hardcoded into if statements • All hands below threshold were folded. • Those with characteristics in top 5th percentile had a higher random chance of being raised

  7. The Turn (continued) • Post-Flop • Neural Net • 5 card input • Inputs the unique id of each card in hand and play • Hidden nodes utilizing sigmoid function • Output using sigmoid function between -1 and 1 • Output values above .3 indicated an action to raise • Below -.3 fold • Else, Call

  8. The Turn (continued)

  9. The Turn (continued) • River • Neural Net • 6 card input • Inputs the unique id of each card in hand and play • Hidden nodes utilizing sigmoid function • Output using sigmoid function between -1 and 1 • Output values above .3 indicated an action to raise • Below -.25 fold • Else, Call

  10. The Turn (continued) • Showdown • Cards analyzed for poker hands such as Flush, Fold, etc. and high card of each returned. • Input nodes for each poker hand. Value is the value of the highest card in hand. • Hidden nodes utilizing sigmoid function • Output using sigmoid function between -1 and 1 • Output values above .5 indicated an action to raise • Below -.2 fold • Else, Call • Neural Net • Player hand analyzed and given as set of input values

  11. The Turn (opponent modeling) • Opponent modeling generalized to fit into previous Neural networks • Opponents action modeling taken into account by Neural nets due to opponents actions changing the weights of neural network to account for their actions. • 2 Factors • Player’s calls • For each raise by opponent, have a higher required confidence to raise again • Players treatment of poker hand values • As player actions change, the confidence values of the Neural nets should change to reflect the modeling of the player

  12. The River (experimentation) • Experiment was developed to overcome short term failure due to bad luck • 1000 dollars given to each player. $10 bet . • Engines initial weights were determined experimentally for each ANNE • Weights were assigned randomly and run for 10000 generations against a set of random poker hands and card tables. Winning gave an expected value of 1, loss -1, and draw 0. • Network was then checked against 1000 random validation hands. • A degree of accuracy was required for each ANNE • Required accuracy lowered as amount of information known decreases. • Once Neural Network was fit level of confidence its results were accepted as the Engines default weights. • Weights continue training during the course of game to profile player. • Tested against a number of human opponents based on skill level to measure success.

  13. The River (experiment results) • Beginners • AI won 82% of games against 50 opponents • Moderate (play at least 1 game a month) • AI won 20% of games against 20 opponents • Advanced (play at least 1 game a week) • AI won 1 game out of 10 against 2 opponents. • Random (never folded) • AI won 63.2% of games out of 1000. • 100% won against Random opponent who could also fold

  14. The Showdown (Conclusion) • The complete reliance on generalized Neural Networks for a system with extreme noise is not a viable option for higher levels of play. • While it did not shown significant advantage over moderate and advanced opponents, it did well against beginning opponents who knew how to play poker but did not do so often. • So much noise…. • While a degree of noise was expected, generalized poker domain’s noise was is in excess of 10-40% (approx.) • This led to extreme cases of overlearning regardless of an extremely low learning rate. • While this was partially handled using confidence, the amount of folds late in the game due to incorrect early predictions led to the machine still failing to moderate opponents. • Was still shown to do well against a random opponent and beginners.

  15. The Showdown (future research) • There are numerous areas for improvement • Current systems adjusts slower than humans • Possible GA implementation instead of Neural Network to avoid direct noise issues. • Statistical analysis and pot odds would help as inputs for neural network.

  16. Questions?

More Related