Cognition and Behavior in Two-Person Guessing Games: An Experimental Study

1. COGNITION AND BEHAVIOR IN TWO-PERSON GUESSING GAMES: AN EXPERIMENTAL STUDYMiguel A. Costa-Gomes Vincent P. Crawford University of York University of California, San Diego April, 2005

2. OVERVIEW I) INTRODUCTION II) EXPERIMENTAL DESIGN III) ECONOMETRIC MODEL IV) SOME FINDINGS

3. INTRODUCTION ACTIONS INFORMATION SEARCHACTIONS & INFORMATION SEARCH

4. INTRODUCTION PREVIOUS STUDIES OF GUESSING GAMES Nagel (1995, AER), Ho, Camerer, and Weigelt (1998, AER), and Grosskopf and Nagel (2001). In their guessing games, n players make simultaneous guesses between lower and upper limits [a, b]. The player whose guess is closest to a target (p) times the group average wins a prize. The structure is public knowledge. The same game is repeated a number of times. Nagel Ho, Camerer, Weigelt Grosskopf and Nagel

5. EXPERIMENTAL DESIGN TWO-PERSON GUESSING GAMES Each player has a lower and an upper limit (both strictly positive), but players are not required to guess between their limits. Guesses outside a player’s limits are automatically adjusted up to his lower limit or down to his upper limit as necessary, Each player also has a target , and his payoff increases with the closeness of his adjusted guess to his target times the other’s adjusted guess.

6. EXPERIMENTAL DESIGN TWO-PERSON GUESSING GAMES Example – Game g: Suppose i guesses 400 --> j guesses 200 --> i guesses 300 --> j guesses 150 --> i guesses 225 (adjusted to 300). Equilibrium is (i guesses 300, j guesses 150) Player i’s payoff is

7. EXPERIMENTAL DESIGN TWO-PERSON GUESSING GAMES Within this structure, we vary the targets and limitsindependently and acrossplayers and games, with targets either both less than one, both greater than one, or mixed (the targets in previous guessing experiments varied only across treatments, or not at all). LOWER AND UPPER LIMITS: (a) [100,500]; (b) [100,900]; (g) [300,500]; (d) [300,900]. TARGETS:(1) 0.5; (2) 0.7; (3) 1.3; (4) 1.5. Example (d4b1):

8. EXPERIMENTAL DESIGN TWO-PERSON GUESSING GAMES Example (d4b1)

9. EXPERIMENTAL DESIGN FEATURES OF TWO-PERSON GUESSING GAMES Reducing n to two allows us to focus on the central strategic problem of predicting the guesses of other players who view themselves as a non-negligible part of one’s own environment. It also eliminates his need to predict how his guess affects an average. Having player-specific limits and targets, moves equilibrium guesses away from the boundaries allowing clearer inferences. Our games are not zero-sum, and have more than two possible payoffs. Consequently, players’ best responses to their beliefs are usually unique within their limits. Deviations are costly.

10. EXPERIMENTAL DESIGN TYPES L1 – plays the best response to beliefs that assign equal probabilities to other's actions. L2 – plays the best response to L1. L3 – plays the best response to the best response to L1. D1 – which does one round of deleting actions that are dominated by pure actions and then best responds to a uniform prior over other's remaining actions. D2 – which does two rounds of deleting actions that are dominated by pure actions and then best responds to a uniform prior over other's remaining actions. EQUILIBRIUM – always plays his equilibrium action. SOPHISTICATED – plays the best response to the probabilities of his partner's actions, which we estimate from the observed frequencies (depends on data).

11. EXPERIMENTAL DESIGN (B) BaselineTreatment -Subjects played 16 guessing games, which are generally asymmetric and dominance-solvable in 3 to 52 rounds, with essentially unique equilibria determined (but not always) by players’ lower limits when the product of their targets is less than one, or their upper limits when the product is greater than one. - We have eight player-symmetric pairs of games (so no need to subdivide subjects into roles), and one pair of symmetric games. - Repeated anonymous random pairing. - No feedback about their own payoffs, their partners’ guesses and payoffs. All subjects framed as Row players and called “You”. Subjects were paid for 5 of the 16 games played. They were paid $0.04 per point. With possible payoffs of 0 to 300 points per game, this yielded payments from $0 to $60. Show-up fee of $8. Average total earnings were $40.21.

12. EXPERIMENTAL DESIGN Computer Interface

13. EXPERIMENTAL DESIGN Click to open a box

14. EXPERIMENTAL DESIGN Click to close the box previously opened

15. EXPERIMENTAL DESIGN Entering a guess

16. EXPERIMENTAL DESIGN (OB) Open BoxesTreatment - is identical to the Baseline treatment except that the 16 games are presented with the targets and limits continually visible, in “open boxes”. - Provides a test of whether interface affects guesses.We find insignificant differences between Baseline and Open Boxes subjects’ guesses. (R/TS) Treatments - are identical to the Baseline treatment, except that each subject is trained and rewarded as a specific type: L1, L2, L3, D1, D2, or Equilibrium. Validates the model of players’ cognitive processes for the L1, L2, L3, D1, D2, and Equilibrium types. - Study the extent to which cognitive processes differ in the Baseline and Robot/Trained Subjects treatments. The distribution of guesses of the Baseline treatment is significantly statistically different from the distribution of guesses of each one of the R/TS treatments.

17. EXPERIMENTAL DESIGN DATA - Guesses (Actions). - Look-ups (box, gaze time - amount of time the box was "open"). Camerer, Johnson, Rymon, and Sen (1993) and Johnson, Camerer, Rymon, and Sen (2002) pioneered the use of MouseLab data (actions, and information search) in games by studying backward induction in extensive-form alternating-offers bargaining games in which subjects could look up the sizes of the “pies” to de divided in each period. Costa-Gomes, Crawford, and Broseta (2001) used MouseLab to study two-person matrix games, with unique equilibria. Several other papers study other kinds of games and single-person decision problems.

18. EXPERIMENTAL DESIGN FEATURES OF THE DESIGN Varying targets and limits within an intuitive structure facilitates teaching subjects the rules and allows them to concentrate on predicting others’ guesses and identifying best responses, which greatly reduces the noisiness typical of initial responses to games. Types’ predicted guesses are more separated than in earlier studies, which mostly used payoff matrix games, in which players have at most 5 possible actions. In particular, the Lk, and Dk types are not separated in previous guessing games experiments, and are only weakly separated in other experiments. Allowing subjects to search within the intuitive common structure of guessing games makes mental models of others easy to express as functions of the targets and limits, as we will see next. As will also be seen next, the L1 type, which seems to best describe most subjects’ actions in earlier studies, requires subjects looking-up elements of the game about the other player (the lower and upper limits). In matrix games, L1 subjects only need to look-up their own payoffs.

19. Computer Interface (as in Costa-Gomes, Crawford, and Broseta (2001))

20. ECONOMETRIC MODEL GUESSES-ONLY ANALYSIS Counting “Beans”  Maximum-Likelihood Error-rate Analysis Specification Test (Overfitting, Omitted types) SEARCH-ONLY ANALYSIS Maximum-Likelihood Error-rate Analysis GUESSES AND SEARCH ANALYSIS Maximum-Likelihood Error-rate Analysis Specification Test (Overfitting)

21. ECONOMETRIC MODEL GUESSES-ONLY ANALYSIS Counting “Beans” 43 of 88 subjects made b/w 7 and 16 of some type’s exact (within 0.5) guesses, 20, 12, and 3 conforming closer to L1, L2, and L3, than to Eq. (8). Maximum-Likelihood Error-rate Analysis: We use a simple “spike-logit” error structure in which, in each game (g), a subject (i) makes his type’s guess exactly with probability , and otherwise makes errors, that follow a logisticdistribution over the rest of the interval between her/his limits. Expected payoff of guess given type-k’s beliefs:

22. ECONOMETRIC MODEL GUESSES-ONLY ANALYSIS Maximum-Likelihood Error-rate Analysis: The probability density function of an error (a guess that differs from type k’s predicted guess) follows a logistic distribution over the rest of the interval between the player’s lower and upper limits. We assume that, given type, errors in guesses are independent across games. Subject i’sguesses related log-likelihood is:

23. SOME FINDINGS GUESSES-ONLY ANALYSIS Maximum-Likelihood Error-rate Analysis (cont.): Estimated Types: 43 L1, 20 L2, 3 L3, 5 D1, 14 Eq., and 3 Sop. Hypotheses Tests: is rejected for all but for 7 subjects. Spike is necessary. is rejected for 34 subjects. Thus, the logit model’s payoff-sensitive errors significantly improve the fit over a spike-uniform model for about 1/3 of the subjects. which corresponds to a random model of guesses within our specification is rejected at the 5% level for all but 10 subjects (6 L1, 2 D1, 1 Eq., and 1 Sop.)

24. ECONOMETRIC MODEL GUESSES-ONLY ANALYSIS “SPECIFICATION TEST” Type estimates could be sensitive to our a priorispecification ofpossibletypes, which might err by omittingrelevant types and/or overfitting by including empirically irrelevant ones. Test is based on the idea of a pseudotype - a pseudotype is constructed from one subject’s guesses in the 16 games. Since we have 88 subjects, we are going to have 88 pseudotypes. (Not all different, since if two subjects’ guesses coincide in all games, their pseudotypes coincide). We can then take some other subject’s guesses in the 16 games, and compute the likelihood of the subject’s guesses, given a pseudotype’s (predicted) guesses. (Not the same subject, since, obviously, the likelihood of a subject’s guesses given the pseudotype based on its guesses is 1!)

25. ECONOMETRIC MODEL GUESSES-ONLY ANALYSIS “SPECIFICATION TEST” The test compares the likelihood of our type estimate to the likelihoods of analogous estimates based on 87 pseudotypes. Omitting – Suppose we had omitted a relevant type, say L2. The pseudotypes of subjects now estimated to be L2 would then outperform the non-L2 types estimated for them, and would also make approximately the same (L2) guesses. We define a cluster as a group of 2 or more subjects such that: (i) each subject’s pseudotype has higher likelihood than the estimated type for each other subject in the group; (ii) subjects’ pseudotypes make “sufficiently similar guesses”. Finding a cluster should lead us to diagnose an omitted type, and studying the common elements of its’ subjects guesses may help to reveal its decision rule. We find 5 clusters with 3, 2, 2, 2, and 3 subjects, respectively (see Table IX).

26. ECONOMETRIC MODEL GUESSES-ONLY ANALYSIS “SPECIFICATION TEST” Overfitting – A subject’s estimated type to be acredible explanation of his behavior should perform at least as well against the pseudotypes as it would, on average, at random. Then for a pseudotype to have higher likelihood than our estimated type it must come first among our 7 types plus itself, which has probability 1/8. The subject’s estimated type has higher likelihood than all but an expected number of 10.75 pseudotypes. (15 type estimates are ruled out on the basis of overfitting – 10 L1, 2 L2, 1 D1, 1 Eq., and 1 Sop., of which 4 L1, 1 D1, 1 Eq., and 1 Sop. already ruled out on the basis of a random model of guesses.)

27. SOME FINDINGS SUMMARY OF REVISED GUESSES-ONLY ANALYSIS Combining our guesses-only estimates with our statistical tests, we say that a guesses-only type estimate appears reliable if: (i) it does significantly better at the 5% level than arandom model of guesses within our specification. (ii)it has higher likelihood than all but at most a random number of pseudotypes. (iii)it is not a member of any cluster. By these criteria, 58 of our 88 subjects’ guesses-only type estimates appear reliable.

28. SOME FINDINGS SUMMARY OF REVISED GUESSES-ONLY ANALYSIS L1: 43 (27 reliably identified, other 16 may be spurious - 5 in clusters + 11 no better than random guesses and/or pseudotypes). L2: 20 (17 reliably identified, other 3 may be spurious - 1 in clusters + 2 no better than random pseudotypes). L3: 3 (1 reliably identified, other 2 may be spurious - in clusters). D1: 5 (1 reliably identified, other 4 may be spurious - 2 in clusters + 2 no better than random guesses and/or pseudotypes). Eq.: 14 (11 reliably identified, other 3 may be spurious - 2 in clusters + 1 no better than random guesses and/or pseudotypes). Sop.: 3 (1 reliably identified, other 2 may be spurious - no better than random guesses and/or pseudotypes). Our findings are close to previous estimates from other kinds of games.

29. ECONOMETRIC MODEL Each type is associated with algorithms that describe how to processinformation about targets and limits into guesses. We use these algorithms as models of each subject’s cognition. We infer a type’sminimal search implications from plausible algorithms for identifying its ideal guess, under conservative assumptions about how cognition affects search, like in our previous paper. Standard assumptions imply a type looks up all freely available information that may affect its beliefs and guesses its target times the mean of its beliefs. The algorithms comprise basic operations and other operations. We assume that basic operations are associated with adjacent look-ups, which can appear in any order but cannot be separated; other operations can appear in any order, and can be separated (although we report the most natural order, we do not insist on it).

30. ECONOMETRIC MODEL L1 Ideal GuessSearch Implications

31. ECONOMETRIC MODEL L2 – 1st Step Her GuessSearch Implications

32. ECONOMETRIC MODEL L2 – 2nd Step My GuessSearch Implications

33. ECONOMETRIC MODEL SEARCH-ONLY ANALYSIS Example of a Look-up Sequence:{ai, pi, pj,ai, bi, aj, pi, bj,…} L2’s Look-up Sequence: Our econometric analysis quantifies compliance with a type’s search implications as the density of the type’s relevant look-up sequence in the subject’s look-up sequence. Since subjects vary widely in where the relevant look-ups tend to be located in their sequences, we filter out some idiosyncratic noise using a binary nuisance parameter called style(“early”or “late”), (assumed constant across games) If , we start at the beginning of the subject’s look-up sequence, and continue until we obtain the type’s complete relevant sequence: Example: {ai,pi, pj,[ai,bi],aj, pi,bj,…} (the L2’s relevant sequence has length six, and the first complete sequence is obtained after eight look-ups, compliance is 0.75).

34. ECONOMETRIC MODEL SEARCH-ONLY ANALYSIS We discretizecompliance into three categories: Let be the probability that subject i has type-kstyle-scompliance c in any given game. Let be the number of games for which subject i has type-kstyle-scompliance c. Subject i’sinformation search related log-likelihood is: 58 out of 71 Baseline subjects’ style estimates are “early”, 10 are “late”, and 3 ties.

35. ECONOMETRIC MODEL GUESSES AND SEARCH ANALYSIS A subject’s type and styledetermine hisinformation search and guess, each with error. We assume that, given type and style, errors in search and guesses are independent of each other and across games. Subject i’sinformation search and guesses log-likelihood is: The model has 6 parameters per subject: error rate , precision , type , style , and two independent probabilities .

36. ECONOMETRIC MODEL GUESSES AND SEARCH ANALYSIS Our subjects’ type estimates change when search is taken into account, for one of two reasons: For some subjects there is a tension between guesses-only and search-only type estimates, resolved in favor of a type other than the guesses-only estimate. For other subjects the type estimate based on guesses-only has 0 search compliance in 8 or more games, and is therefore ruled out by a priori constraints. When the guesses-and-search type estimatediffers from the guesses-only estimate, we favor the former but require it to pass the analogs of the guesses-only criteria. We say that a guesses-and-search type estimate is reliable if: (i) it does significantly better at the 5% level than a random model of guesses and search within our specification. (ii) The guesses-only part of its likelihood has higher likelihood than all but at most a random number of pseudotypes. (iii)it is not a member of any cluster.

37. SOME FINDINGS SUMMARY OF GUESSES AND SEARCH ANALYSIS The search and guesses analysis: - Confirms the reliability of 39 of our 46 Baseline subjects whose guesses-only type estimates appear reliable. - It also reliably identifies 2 subjects as L1 who had appeared reliably identified as L2. - Confirms the reliability of 1 L1 and 1 L2 subjects whose guesses-only estimates were inconclusive. - It calls into question the type estimates of 4 subjects who had appeared reliably identified: 1 L1, 1 L2, 1 D1, 1 Eq. In short, 54 of 88 subjects’ types are identified with confidence: 29 L1, 14 L2, 10 Eq., 1 L3, 1 Sop. (last was OB, and might not have survived monitoring search).

38. SOME FINDINGS IDENTIFIED TYPES AND DECISION TIMES: average time per guess according to subjects’ estimated types (based guesses and information search). B Treatment: L1 (22): 63.7 sec. L2 (13): 82.1 sec. Eq. (8): 117.2 sec.

39. CONCLUSIONS Reports an experiment that elicits subjects’ initial responses to 16 dominance-solvable two-person guessing games, monitoring their searches for hidden but freely accessible parameters of the games along with their guesses. The design yields strong separationof the guesses and information searches implied by leading decision rules. Cognition is studied at the individual level as in Costa-Gomes, Crawford, and Broseta (1998) UCSD-DP. Of our 88 Baseline and OB subjects, 54 can be reliably identified as one of our types based on guesses and search. Because our types specify precise guesses in large strategy spaces, the identifications show that those subjects have accurate models of the games and acted as rational, self-interested expected-payoff maximizers. Lk types are overwhelmingly predominant. This lends support to the leading role given to iterated best responses in informal analyses of strategic behavior.

41. EXPERIMENTAL DESIGN TREATMENTS BASELINE (B) vs. OPEN BOXES (OB): provides a test of whether interface affects guesses and decision time. Time to make a guess increases (from 74.50 to 91.87 seconds). The distribution of guesses are not significantly statistically different. BASELINE (B) vs. ROBOT/TRAINED SUBJECTS (R/TS): validate the model of player’s cognitive processes for the L1, L2, L3, D1, D2 and Equilibrium types; study the extent to which cognitive processes differ in the Baseline and Robot/Trained Subjects treatments. Time to make a guess decreases (from 74.50 to ??? seconds). The distribution of guesses of the Baseline treatment is significantly statistically different from the distribution of guesses of each one of the R/TS treatments.