Can Negotiation Breakdown Probabilities of Laissez-Faire Agents Be Derived A Priori?

1. Can Negotiation Breakdown Probabilities of Laissez-Faire Agents Be Derived A Priori? R. Loui, R. Ratkowski, J. RosenDepartment of Computer Science and EngineeringWashington UniversitySt. LouisUSA r.p.loui AIVR UIUC

2. Trailers/Previews • DataMining on OC192 Data Streams (10Gbps) • A.k.a. "Streaming AI" • With John Lockwood (from UIUC), PI • 1M x speedup of classified intelligence task • Better performance than available software • Better performance than SVM methods • Related to FPgrep, FPsed, FPawk patent US 7093023 • Methods, systems, and devices using reprogrammable hardware for high-speed processing of streaming data to find a redefinable pattern and respond thereto • W/Moshe Looks, papers on hierarchical streaming clustering r.p.loui AIVR UIUC

3. Trailers/Previews • Scripting Languages and The New Programming Pragmatics • http://www.cse.wustl.edu/~loui/praiseieee.txt • The real shock is that academia continues to reject the sea change in programming practices brought about by scripting. • Scripting was not enervating but was actually renewing: programmers who viewed code generation as tedious and tiresome … viewed scripting as rewarding self-expression or recreation. • I personally believe that CS1 java is the greatest single mistake in the history of computing curricula. • Linguists recognize something above syntax and semantics, and they call it "pragmatics". We are entering an era of comparative programming language study when the issues are higher-level, social, and cognitive too. r.p.loui AIVR UIUC

4. Trailers/Previews • Moshe Looks, D.Sc. 06 (expected) • Externally advised by David Goldberg (UIUC) and Martin Pelikan (formerly UIUC) • COMPETENT PROGRAM EVOLUTION • My thesis is that the properties of programs and program spaces can be leveraged as inductive bias to reduce the burden of manual representation-building, leading to competent program evolution. • The central contributions of this dissertation are • a view of the requirements for competent program evolution, and • the design of a procedure, meta-optimizing semantic evolutionary search (MOSES) r.p.loui AIVR UIUC

5. Trailers/Previews • Dynamics of Rule Revision and Strategy Revision • With M. Looks, B. Cynamon (U Chicago), A. Schiller (Princeton) • A.k.a. Legislature vs. Population games • H.L.A. Hart: There is a limit, inherent in the nature of language, to the guidance which general language can provide. • Abridgement = Projection of a veridical utility function • Scenario extinction • Sheep, Weasels, and Gorillas r.p.loui AIVR UIUC

6. Today's Abstract We present a different AI model of negotiation where agents are driven by dynamic expectations (there is no solution concept and there is no recursive modeling of beliefs). We require two assumptions to paint a new picture: (1) there is an empirically observable objective probability of breakdown that is monotonic (at some granularity) in elapsed time since last progress; (2) there is a nonstandard utility attached to the act of unilateral breakdown: a process utility that models the satisfaction of breaking down on a non-cooperative negotiating partner. This is a procedural fairness adjustment, not the substantive distributive fairness effect that has been trendy in the economics literature. We observe the variety of behaviors that can be generated by agents constructing action according to such Pessimism-Punishment (PP) negotiation models. We define a laissez-faire path for two PP agents starting in a given position and the proper calibration of their breakdown probabilities conditioned only on position. Finally, we discuss what iterative process could be used to reduce a priori miscalibration of breakdown probabilities. r.p.loui AIVR UIUC

7. Negotiation Behavior • Social Psychology: Dean Pruitt • Logrolling issues • Management Sci: Howard Raiffa / Max Bazerman - Margaret Neale • Integrative agreement • Law: Roger Fisher – William Ury – Bruce Patton • Principled negotiation • Artificial Intelligence: • Problem solving: R Davis – R Smith / V Lesser • Shared planning dialogue: S Carberry / G Ferguson – J Allen • Non-ideal game theory: E Durfee / S Rosenschein / T Sandholm • Argumentation: K Sycara / S Parsons – C Sierra – N Jennings • Economics: John Nash / Ariel Rubinstein • Solution concept • Equilibrium r.p.loui AIVR UIUC

8. Negotiation Behavior: Equilibrium • Mathematical curiosity (cf. Axelrod) • Starts with "Solution concept (I or II)": reduction of uncertainty to a distinct outcome • Epistemologically far-fetched • Empirically ridiculous • Philosophically indefensible • Useless in the design of negotiating agents r.p.loui AIVR UIUC

9. Negotiation Behavior: Equilibrium • Mathematical curiosity (cf. Axelrod) • Starts with "Solution concept (I or II)": reduction of uncertainty to a distinct outcome • Epistemologically far-fetched • Empirically ridiculous • Philosophically indefensible • Useless in the design of negotiating agents r.p.loui AIVR UIUC

10. Negotiation Behavior: AI • A place for language / argument • A place for introspection on utilities • A range of interesting & reasonable behaviors • Participation in the process of negotiation: • Exchanging proposals • Reacting to proposals r.p.loui AIVR UIUC

11. My Theory Pt. I • Observation: parties to a negotiation (can) construct a probability distribution over potential settlements r.p.loui AIVR UIUC

12. Party 1'saspiration Party 2'saspiration r.p.loui AIVR UIUC

13. Party 1'sproposals at t Party 2'sproposals at t r.p.loui AIVR UIUC

14. inadmissible(dominated)at t inadmissible(dominated)at t r.p.loui AIVR UIUC

15. In black: admissiblesettlementsat t(probabilityof agreement Is non-zero) r.p.loui AIVR UIUC

16. 1's aspiration 2's aspiration r.p.loui AIVR UIUC

17. Breakdown (BATNA) r.p.loui AIVR UIUC

18. Breakdown (BATNA) r.p.loui AIVR UIUC

19. Breakdown column Breakdown row r.p.loui AIVR UIUC

20. Breakdownwould occurhere (BATNA) r.p.loui AIVR UIUC

21. 1's security level 1 would rather breakdown 2's security level 2 would rather breakdown r.p.loui AIVR UIUC

22. Prob(bd) = ? Eu1|s = 51 Eu2|s = 49α +54(1-α) r.p.loui AIVR UIUC

23. My Theory Pt. I • Observation: parties to a negotiation (can) construct a probability distribution over potential settlements r.p.loui AIVR UIUC

24. My Theory Pt. I • Observation: parties to a negotiation (can) construct a probability distribution over potential settlements • What kind of probability? r.p.loui AIVR UIUC

25. My Theory Pt. I • Observation: parties to a negotiation (can) construct anobjective, empirical or epistemological*(NOT SUBJECTIVE*) probability distribution over potential settlements from past experience in similar settings • OBJECTIVE: • Constructed from data • Agree on P, given K • Not committed to P(a) until queried about P(a) • SUBJECTIVE: • "Bayesian" (but not necessarily what AI people mean) • Can change by new prior, new conditioning, or shift in feelings • Total and consistent prior to querying r.p.loui AIVR UIUC

26. My Theory Pt. I • Observation: from a probability distribution over potential settlements, there is an expected utility given settlement • Observation: there is a probability of breakdown p(bd) r.p.loui AIVR UIUC

27. P(bd) gap probability of break down r.p.loui AIVR UIUC

28. My Theory Pt. I • Observation: from a probability distribution (at t) over potential settlements, there is an expected utility given settlement (at t) • Observation: there is a probability of breakdown pt(bd) r.p.loui AIVR UIUC

29. My Theory Pt. I • Observation: At t, calculate 1. An expected utility given settlement (Eut|s) and 2. An expected utility given continued negotiation,Eut = (Eut |s) (1 - pt(bd)) + u(bd) pt(bd) • Definition: Rationality requires the agent, at t, to: 1. Extend an offer, o, if Eut < u(o) 2. Accept an offer, a, if Eut < u(a), a  offers-to-you(t) 3. Break down unilaterally if Eut < u(bd) r.p.loui AIVR UIUC

30. My Theory Pt. I • Why not iff? • Extend an offer, o, if Eut < u(o) • Withhold an offer?, o, if Eut > u(o) • There may be other reasons for acting earlier • Constructivism: • Multiple ways of constructing probability • Multiple ways of deriving/justifying/motivating action r.p.loui AIVR UIUC

31. My Theory Pt. I • Empirical Observation: At sufficient granularity, p(bd) is decreasing in the time since last progress r.p.loui AIVR UIUC

32. My Theory Pt. I PessimismFor sufficiently large Δ, where LP(t0) denotes last progress at t0 pt+Δ(bd | LP(t0)) > pt(bd | LP(t0))What is progress?A non-trivial offer by the other partyWhat does this mean?(at some granularity, the past record implies that:)If there are no offers, the probability of breakdown rises r.p.loui AIVR UIUC

33. My Theory Pt. I PessimismLinear Pessimismp(bd | NP(t)) = π tExponential Pessimismp(bd | NP(t)) = 1 - e-πt Delayed Linear Pessimismp(bd | NP(t)) = π max(0, t - t0) Whatever fits the empirical record! r.p.loui AIVR UIUC

34. Pessimism causes Eu to fall Next offer is made at this timeand prob(bd) resets to 0 Expectation starts to fall again r.p.loui AIVR UIUC

35. offers reciprocated offers r.p.loui AIVR UIUC

36. Agreement reached as Eu < u1 r.p.loui AIVR UIUC

37. Whenever u(acc) > security, acceptance occurs before breakdown! Best offer received security r.p.loui AIVR UIUC

38. Would you accept an 11-cent offer if yoursecurity were 10-cents? Best offer received security r.p.loui AIVR UIUC

39. My Theory Pt. II • Observation: You wouldn't accept 11¢ over 10 ¢ security, nor 51 ¢ over 50 ¢ security • Observation: You wouldn't let your kid do it • Observation: Your Mother wouldn't let you do it • Observation: Your lawyer wouldn't let you do it • Observation: Your accountant wouldn't let you do it • Proposition: We shouldn't automate our agents to do it r.p.loui AIVR UIUC

40. My Theory Pt. II • Question: Isn't this an issue of distributive justice • Answer: Substantive fairness is trivial to model by transforming utilities • Observation: There may be a procedural fairness issue r.p.loui AIVR UIUC

41. My Theory Pt. II • Procedural fairness: • the more the other party withholds progress, the more you will punish • When the other party resumes cooperation, you are willing to forgo punishment r.p.loui AIVR UIUC

42. My Theory Pt. II Resentmentu(bd) = security + resentment(t) What is resentment?1. Dignity2. Pride3. Investment in society4. Protection against non-progressive manipulators 5. A GENUINE source of satisfaction: non-material, transactional, personal(?), transitory(?) r.p.loui AIVR UIUC

43. My Theory Pt. II Resentmentut(bd) = security + resentment(t) = u(bd) + r(t) for NP(t), non-progress for a period tWhat is resentment?6. Attached to a speech/dialogue act: BATNA through breaking down vs. BATNA through agreement7. A nonstandard utility (process utility)8. Specific or indifferent (I-bd-you vs. you-bd-me) r.p.loui AIVR UIUC

44. My Theory Pt. II Resentmentlinear resentmentr(t) = ρtsigmoid resentmentr(t) = rmax(2/(1+e-ρt) -1) You either feel it or you don't – you can't fake it! r.p.loui AIVR UIUC

45. Eu never falls to u1 r.p.loui AIVR UIUC

46. Actually accepts becauseresentment resets with progress Nontrivial progess Resentment resets to zero each time there is progress r.p.loui AIVR UIUC

47. Agent breaks down before accepting Resentment might not reset to zero if there is memory r.p.loui AIVR UIUC

48. P&P Agents Pessimism + Punishment"purely" probablistic r.p.loui AIVR UIUC

49. Variety of Behaviors • Agent can make a series of offers, responds to offers • Agent can wait, then offer, accept, or break down • Agent can accept, offer, or break down immediately • Agent can offer before accepting and vice versa • Agent can breakdown before accepting and vice versa • Agent can offer before breaking down and vice versa • Agent can be on path to breakdown, then on path to acceptance • because received offer changes Eu or resentment • because extended offer changes Eu • I wouldn't use this as my agent on ebay quite yet… r.p.loui AIVR UIUC

50. (Assumes no progress) Linear pess/linear specific pun low-valued ρ high-valued ρ r.p.loui AIVR UIUC