Theory & Experimental Design

1. Theory & Experimental Design Part II

2. FTT & Political Science • Two generations (although overlap time wise): • First – testing equilibrium & non-equilibrium predictions of social choice theory • Second – testing more applied models, typically with greater institutional (political) detail. • Mirror evolution of formal models in discipline

3. First Generation Tests on Elections & Committees – Tests of Spatial Voting Models • This work is reviewed in • McKelvey, Richard D. & Peter C. Ordeshook, 1990, “A Decade of Experimental Research on Spatial Models of Elections and Comittees,” in James M. Enelow & Melvin Hinich, eds., Advances in the Spatial Theory of Voting, Cambridge: Cambridge U. Press.

4. First Generation Tests on Committee Voting • Goal of initial work on committees – • see extent disequilibrium really happened in lab. • Not so easy to avoid equilibrium creating conditions not suggested by theory. • For example, if subjects know (or suspect) experiment has an “end” or consider time spent as costly, may be motivated to agree on a choice.

5. First Generation Testson Generic Voting Games • While fundamental – largely become province of social choice theorists & theoretical side dominates • Main emphasis what is necessary get equilibrium, considering axioms of social choice theory & implications. • Social Choice & Welfare (both society & journal) – main avenues, mainly normative

6. Example of “Modern” “First Generation Like” Voting Experiment • “Voting Games and Computational Complexity,” Glenn W. Harrison & Tanga McDaniel, working paper

7. Voting Rule Experiment • Authors confront Gibbard-Satterthwaite Theorem – loosely stated – only voting rule that is “strategy proof” for all possible preference profiles is dictatorial. • “strategy proof” – when choices people make under voting rule are truthful – reflect true preferences. • Consider an example – 3 choices, x, y, & z, & 3 types of voters, A (40 voters), B (20 voters), & C (40 voters)

8. Example of Strategic Voting Who will win this election? Not obvious. If voters A, B, & C all voted for their most preferred choice (truth telling), & we use majority rule, then a tie between x & z.

9. Example of Strategic Voting But is that what we would want as a society? Normatively? Condorcet argued that most preferred choice is one who would Defeat or tie all others in pairwise (binary) contests.

10. Example of Strategic Voting Who is the Condorcet winner? Binary choice between x & y, A votes for x, B & C for y, y wins. Binary choice between y & z, A & B for y, C for z, y wins. y is the Condorcet winner.

11. Example of Strategic Voting Majority rule is definitely not “strategy proof” & in fact, normatively we might prefer that voters “strategize” & positively expect them to – Making Votes Count

12. Goal of Voting Experiment • Harrison & McDaniel contend that some voting rules may be more difficult (in a computational sense) to manipulate than others. • They want to test their hypothesis in the lab.

13. Voting Rule Experiment: Goal • H & M test a voting rule in laboratory they believe is • Easy to explain to subjects • Easy to implement • Difficult for subjects to strategically manipulate (because of unspecified cognitive limits)

14. Voting Rule Experiment: Goal • Theory testing or fact finding? • While voting rule is “script” as in other formal theory experimental tests, H & M expect something not modelling (cognitive limits of subjects) is important & are “fact finding.” • Also, some “policy pre-testing” • Nevertheless – design of experiment is “scripted” by theory

15. Proposed Voting Rule • One voting rule that might be good is to always select Condorcet winner – in fact this has been proven to be strategy proof.

16. Condorcet Winner Voting Rule Can voters manipulate this voting rule getting a more preferred outcome by misrepresenting their preferences?

17. Condorcet Winner Voting Rule Who is the Condorcet winner now?

18. Proposed Voting Rule • problem –Condorcet winner not always exsit. • Extension of Condorcet rule by Young – find non-cyclical ranking with most support of voters. • Turns out solution found by solving linear programming problem • Condorcet Consistent Voting Rule – basically chooses Condorcet winner if one exists, if not, non-cyclical ranking with greatest support.

19. Condorcet Consistent Voting Rule Who would win? In this example C voters’ ranking would be maximal, note similarity w/ dictatorship in this example -- but rule not same as dictatorship.

20. Computation & Voting Rule • strategic voting under Condorcet Consistent voting rule requires complex computations when # of alternatives is large – • for n alternatives there are n! non-cyclic rankings. • Thus while not strategy proof in theory, contend behaviorally incentive compatiblewith truth telling (non strategic voting). • Point of Experiment – is this true?

21. Inducing Preferences in Voting Experiments • Usually money • Tell subjects will pay based on which alternative, x,y,z is chosen & have subjects vote. • Can measure extent of strategic voting by comparing choices to induced preferences.

22. “Home Grown” Preferences • Harrison & McDaniel use “home grown” preferences • Subjects given list of CD’s, grouped in categories • Subjects vote over which category for group, then each picks a CD from the list of 10 in category • Difficulty – how measure strategic voting?

23. How Measure Strategic Voting? • Use “control” treatment – subjects choose CD’s under a “random dictator” voting rule • But to be sure instruct subjects in this treatment only on advantages of telling truth – non neutral instructions • Note importance of random assignment. • Are home grown preferences desirable here?

24. Heterogeneity of Preferences • strategic voting “harder” when preferences of voters more heterogeneous. • Vary heterogeneity : • Simple treatment – music categories of Jazz/Easy Listening, Classical, R&B, Rock, C&W. • Complex treatment – music categories of Jazz/Easy Listening, Classical, Heavy Metal, Rap, C&W. • Contend most subjects prefer R&B or Rock.

25. Table 1: Musical Categories & CD’sCategory A: Jazz/Easy Listening • (1) Najee, “Share My World” • (2) Kenny G, “Breathless” • (3) Art Porter, “Undercover” • (4) Russ Freeman & the Rippingtons, “Sahara” • (5) Tony Bennet, “Unplugged” • (6) George Howard, “A Home Far Away” • (7) Enigma 2, “The Cross of Change” • (8) Billy Joe Walker, “Life is Good” • (9) Barry Manilow , “Singing in the Big Bands” • (10) Nat King Cole, “The Greatest Hits”

26. Category B: Classical • (1) John Williams & the Boston Pops Orchestra, “It Don’t Mean a Thing if it ain’t got that Swing” • (2) Vivaldi, “The 4 Seasons” Gil Shattam Orpheus: Fritz Kreisler. • (3) Mahler, Symphony #5. The New York Philharmonic: Leonard Bernstein. • (4) Yo Yo Ma, The New York Album. Baltimore Symphony Orchestra: David Zinman. • (5) Handel, “Messiah” Atlanta Symphony Orchestra & Chamber Chorus: Robert Shaw. • (6) Cecilia Bartoli, “Mozart Portraits” Vienna Chamber Orchestra: György Fischer. • (7) Van Clyburn in Moscow. Brahms Rachmaninoff. M oscow Philharmonic Orchestra: Kiril Konorashin. • (8) Kiri, “Her Greatest Hits Live” London Symphony Orchestra: Steven Barlow. • (9) Tchaikovsky, “Nutcracker” London Symphony Orchestra: S ir Charles Mackerras. • (10) The Best of Wolfgang Amadeus Mozart. Academy of St. Martin-in-the-fields: Neville Marriner.

27. Category C: Heavy Metal • (1) Pantera, “Far Beyond Driven” • (2) Queensryche, “Promised Land” • (3) Magadeth, “Youthanasia” • (4) Motley Crüe, “Motley Crue” (featuring Hooligan’s Holiday) • (5) Mother Tongue “Mother Tongue” • (6) Obituary, “World Demise” • (7) Jackyl, “Push Comes to Shove” • (8) Alice in Chains, “Jar o f Flies” • (9) Alice Cooper, “The Last Temptation” • (10) Cinderella, “Still Climbing”

28. Category D: Rap • (1) Pete Rock and CL Smooth, “The Main Ingredient” • (2) Lighter Shade of Brown, “Layin in the Cut” • (3) Craig Mack, “Project: Funk Da World” • (4) Ghetto Mafia, “Draw the Line” • (5) Common Sense, “Resurrection” • (6) Stevie B, “Funky Melody” • (7) Salt-n-Pepa, “Very Necessary” • (8) j. Little, “Puttin’ it Down” • (9) Celly Gel, “Heat 4 Yo Azz” • (10) Hammer, “The Funky Headhunter”

29. Category E: Country & Western • (1) Garth Brooks, “In Pieces” • (2) George Ducas, “George Ducas” • (3) Shenandoah, “In the Vicinity of the Hearth” • (4) Chris Ledoux, “Haywire” • (5) Willie Nelson, “Healing Hands of Time” • (6) Vince Gill, “When Love Finds You” • (7) Pam Tillis, “Sweetheart’s Dance” • (8) Noah Gordon, “I Need a Break” • (9) Rodney Crowell, “Let the Picture Paint Itself” • (10) Ricky Lynn Gregg, “Get a Little Closer”

30. Information to Subjects • Varied information gave subjects: • Information Treatment – specifics of CC voting rule spelled out to subjects & examples supplied • No Information Treatment – subjects only told that the social ranking chosen would be the one which would most likely receive the support of a majority of the voters. • Why vary this? • Policy Pre-testing . . .

31. Experimental Design

32. Results of Voting Rule Experiment • Gist – found under simple preference profile significant difference between rankings in CC & RD. • No significant difference between rankings in CC & RD under tough preference profile. • Moreover, information matters only in simple preference profile experiments. • Results support argument that when preferences are “tough” to figure out, CC does elicit truth telling.

33. First Generation Research: Summary • Voting Rule Experiment example of an experimental test of research from social choice theory – • like first generation of FTT in political science. • While experiment interesting & results important, most political scientists not think FTT as in political science today. • Why?

34. Impact of Disequilibrium Results on Formal Theory in Political Science • Economists can frequently start from wellaccepted equilibrium models, & then do comparative statics bystandard techniques. • Political theory not have well acceptedequilibrium models to start from. • Theory must incorporatedetails of situation.

35. Impact of Disequilibrium Results on Formal Theory in Political Science • explicitly model role of • information, • repetition • & institutions • usually accompanied by increasing use of non-cooperative game theory, • incompleteinformation, • & explicit specification of extensive forms. • The “New Institutionalism”

36. Impact of Disequilibrium Results on Formal Theory in Political Science • Anothertrend – evolutionary & agent based models. • Different view of behavior – individuals programmed to behave in certain ways & only changebehavior through replacement or imitation (e.g. Bendor, Diermeier, & Ting) • Complex processes literature, etc. • Difficult for laboratory experiments – long term processes of evolution unlikely in single experiment.

37. Experiments on Classic Games & Political Science • Classic games often building blocks in formal work in political science. • Example – bargaining games ultimatum & dictator games add together to get Baron/Ferejohn legislative bargaining game • Example – voting turnout like public good/pd combined

38. Turnout as Combined Public Good/PD • Think of a two candidate election w/ two groups of supporters (teams or political parties). • Each group member individually decides whether or not to vote, paying individualized cost to voting. • Group with most voters wins group payoff distributed equally to all group members whether voted or not. • Team turnout game combination of public good game (within a team) & a prisoner’s dilemma game (between teams).

39. Turnout Experiment: Theory • Turnout modelled this way by Palfrey & Rosenthal, 1983, “A Strategic Calculus of Voting,” Public Choice & in APSR, 1984. • Showed equilibria exist with positive turnout. • Model tested experimentally by Schram and Sonnemans, International Journal of Game Theory, 1996. • Good example of ways formal theory testing works with more complex game.

40. Turnout Experiment: Design • Subjects split into 2 groups of 6 each, labelled yellow & blue. • Each subject had to decide whether to buy an imaginary disc. • Price of a disc was common knowledge & equal for everyone. • # of discs bought by group determined payoffs. • Payoffs equal for everyone within a group. • Repeated for 20 periods.

41. Turnout Experiment: Design • Two payoff schedules, representing winner take all (WIN) & proportional representation (PR). • In WIN, each group member bought most discs received payoff of 2.5 Dutch gilders & other group received zero, ties broken randomly. • In PR, payoff was proportional to turnout within group – # of discs bought in one’s group was divided by total number bought & multiplied by 2.22 Dutch gilders. • Price of disc – 1 gilder (WIN), 0.75 gilders (PR)

42. Turnout Experiment: Predictions • Nash equilibria of games in pure strategies (one shot): • In PR, one disc bought by each group. • In WIN, 6 discs bought by each group. • (turnout theoretically higher in WIN, why? – relationship prediction) • Quasi-symmetric mixed strategy equilibria: • In PR, all subjects buy a disc with probability 0.098. • In WIN – two equilibria • All subjects buy with probability 0.051 • All subjects buy with probability 0.949

43. Turnout Experiment: Results • Find comparative static predictions are supported (relationship predictions) • But point predictions not. • Schram & Sonnemans in Journal of Economic Psychology, 1996 compare analysis with hypotheses derived from a model of turnout that incorporates group pressure as explanation of turnout. • Find some support for group model, although experiment not an explicit test of model (since designed to test Palfrey/Rosenthal)

44. FTT & Experimental Design • Game directly from mechanics of Palfrey/Rosenthal model (script). • not “described” to subjects as voting situation with candidates as in election (frame). • Why? • Advantage – • experiment tests “theory” of participation without “baggage” subjects may bring about voting as an act – • can serve as “baseline” results to experiments where act is described as voting w/ candidates etc. • Disadvantage – decreases “external validity”? • Is this deception?

45. FTT & Experimental Design • Palfrey/Rosenthal – • general model without specific values – for experiment specific values must be set for payoffs & cost of participation. • Experimenters must solve model for equilibrium predictions with specific values. • Palfrey/Rosenthal model had no PR, only WIN • experimenters take model, solved it for particular values, plus made modifications to model.

46. FTT & Experimental Design • Unless working with simple formal theory, usually modifications/limitations needed in testing a formal theory • Difficult to truly fit theory as originally devised even in laboratory with a lot of control • Still often easier than w/o naturally occurring data . . .

47. FTT & Experimental Design • Probably more difficult to fit theory as script for experiment for political science models than for economists. • Why? more institutional detail, more applied, & thus more bells & whistles to either try to put in or simplify.

48. Fitting Design to Theory: Example of Difficulties • choice variable often a continuous – makes solving models easier & sometimes necessary for solutions. • two candidate competition over a unidimensional policy space, assume candidates can choose any point –an infinite choice set. • But suppose wanted to test this – if tell subject choose number between 1 & 10 & tell subject any fraction is acceptable, is subject truly “thinking” continuously?

49. Experiments & PTT

50. Theory Testing: Review • Psychological or Social Psychological Theories • Typically non formal, i.e. • assumptions underlying theory stated verbally • hypotheses about variables posited. • Equilibrium predictions rarely derived. • Usually decision-theoretic • Focused often on process of choice & internal process of mind.