1 / 40

Game Theoretic Aspect in Human Computation

Game Theoretic Aspect in Human Computation. Presenter: Chien-Ju Ho 2008.12.30. Outline. Human computation Examples Rethinking the ESP Game Game theoretic analysis Background knowledge of game theory Example: PhotoSlap General game modeling Classification of human computational games

chiko
Télécharger la présentation

Game Theoretic Aspect in Human Computation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Game Theoretic Aspect in Human Computation Presenter: Chien-Ju Ho 2008.12.30

  2. Outline • Human computation • Examples • Rethinking the ESP Game • Game theoretic analysis • Background knowledge of game theory • Example: PhotoSlap • General game modeling • Classification of human computational games • Discussion of the general game modeling

  3. ESP Game & Peekaboom

  4. Tag a Tune

  5. GWAP – Games With A Purpose • http://www.gwap.com/gwap/

  6. Is the Game Design Rational?

  7. Rethinking the ESP Game • Technical report of Microsoft Research in September 2008. • Target: Google Image Labeler

  8. Rethinking the ESP Game • Shortcomings of the ESP Game • Easy words first • “building” are preferred than ”terraced house” • Tendency to match on colors • Over 10% of off-limit labels are colors • Information redundancy • 81% of images labeled “guy” are labeled “man” • 68% of images labeled “clouds” are labeled “sky”

  9. Rethinking the ESP Game • An auto-playing robot • Analyze the off-limit words • No analysis of the images • Probability of matches • 81% for images with at least one off-limit words

  10. Game Theoretic Analysis In Human Computation

  11. Background Knowledge of Game Theory

  12. Prisoner’s Dilemma

  13. Normal-form Game • The elements of a normal-form game • Players: (prisoner A, prisoner B) • Strategies: (stay silent, betray) • Payoff: (sentences)

  14. Nash Equilibrium • Strategy profile: a combination of strategies. ex: (A: betray, B: betray) • Each player has no incentive to change her action under the assumption that all the other players do not change their actions.

  15. Extensive Game • Sequential structure of decision-making Players Strategies in out Google: $ 1b MS: $ 2b acquiesce fight Payoff Google: $ 2 b MS: $ 1b Google: $ 0 MS: $ 0

  16. Subgame Perfect Equilibrium • Play in each subgame is a Nash equilibrium. • (in, acquiesce): A subgame perfect equilibrium

  17. PhotoSlap • A Multi-player game for face recognition

  18. The Game Rule • Slapping on matching pairs Slap The first player who slapped get 100 points.

  19. The Game Rule • Slapping on matching pairs Slap The first player who slapped get 100 points.

  20. The Game Rule • Objecting on un-matched pairs Slap Objection! The first player who object get 100 points. The player who slapped lose 100 points.

  21. The Game Rule • Objecting on un-matched pairs The first player who object get 100 points. The player who slapped lose 100 points.

  22. The Game Rule • Setting traps on matching pairs Trap Slap Objection! The player who set the trap get 50 points. The player who objected lose 1000 points.

  23. The Game Rule • Setting traps on matching pairs Get 50 points if the trap pair is slapped. Lose 50 points if the trap pair appears but not slapped.

  24. Game Theoretic Analysis For each photo pair in the trap page For each photo pair in the game page Player node Chance node The player who sets the trap The player who objects first The player who slaps first Payoff of player 1 Payoff of (player 2, player 3)

  25. Subgame Perfect Equilibrium • A solution concept in extensive game

  26. Human Computation Modeling

  27. Human Computation • Question and Answer • Human as the algorithm Input Output

  28. Human Computation • Make the process fun • Psychology analysis • Implementation dependent

  29. Human Computation • Ensure the correctness of the output • Motivate helpful behavior • Guarantee high probability of correctness even if some malicious players exist • Apply social validation • Gain points when results match • Check the results produced by different players

  30. Game Templates (1) • Output agreement

  31. Game Templates (2) • Inversion problem

  32. Game Templates (3) • Input agreement

  33. Verification Mechanism • Simultaneous Verification • Sequential Verification

  34. Simultaneous Verification • Coordination Game • Multiple equilibria • Equilibrium selection according to the player’s belief • In the ESP Game • Players would choose the words • More relevant to the image • More generic

  35. Sequential Verification

  36. Why Game Theory • A theory for understanding the incentive structure and player strategies of games • Widely used in different domains, including economics and social science

  37. Open Issues • Extension of N-player games • Effect of changing incentive structure • Consideration for malicious players • Different validation mechanisms

  38. Research in Human Computation • Applications and information integration • Repeated labeling problem • Human computation as the correctness measure

  39. References • von Ahn, L. and Dabbish, L. 2008. Designing games with a purpose. Commun. ACM 51, 8 (Aug. 2008), 58-67. • Ho, C., Chang, T., and Hsu, J. Y. 2007. PhotoSlap – a multi-player online game for semantic annotation. AAAI07 (Jul. 2007). • Weber, I., Robertson, S. and Vojnović, M. 2008. Rethinking the ESP Game. Technical Report, Microsoft Research (Sep. 2008).

  40. Thanks for your listening!

More Related