Distributed Search by Agents with Personal Preferences

1. Distributed Search by Agents with Personal Preferences AlonGrubshtein Lessons learnt from applying distributed constraint reasoning to “realistic” agents with personal preferences

2. Before we begin…

3. In this talk: Constraint Reasoning Distributed Computing Distributed Constraint Reasoning Multi Agent Systems

4. Sometime back in 2006 You can even write programs for it… Lets write a distributed agent to automate meeting coordination Check out this great phone I got I can use it to work on my calendar!!! Who needs a computer with such phones?

5. Constraint reasoning (centralized) A Constraint Reasoning problem: • Variables • Domains • Constraints (relations) A solution concept (target objective)

6. Examples

7. What’s in a constraint? Two important classes of problems: • Constraint Satisfaction (CSP) • Constraint Optimization (COP) A satisfying assignment A minimal cost assignment

8. Constraint algorithms How do we find a solution? • Enumerate feasible outcomes • Backtracking / Branch and Bound • Intelligent backtracking • Pre processing, forward checking and heuristics • Local search algorithms

9. From centralized to distributed The problem itself is distributed across computational nodes – agents: • Privacy • “Difficulty”

10. Constraint reasoning (distributed) Distributed Constraint Reasoning (DCR) problem: • Agents • Variables • Domains • Constraints (relations) DCSP / DCOP

11. From centralized to distributed • Computation on separate entities • Communication via messages • Each agent knows only a small portion of the problem • Allows for parallel computation DISTRIBUTED =/= PARALLEL

12. DCR algorithms

13. Local Search for “real” problems • Computationally hard • Simplistic myopic algorithms (“local search”/“adaptiveheuristics”) • Example, DSA: • Pick a random assignment • While (stop condition): • Send assignment to all neighbors (receive) • If can improve local state by changing assignment:change with probability p

14. A simple MAS example Anna Anna Anna E E E M M M Alice Alice Alice 3 5 , 3 5 1 , 0 1 0 M M M Coordinating a meeting (e.g., seminar): • Two alternatives: Morning or Evening • More participants – better • Prof. Lynn does not care when • If students disagree - morning • Alice prefers morning • Anna prefers evening 0 , 2 2 0 2 4 2 , 4 E E E Prof. Lynn Alice Anna

15. Solving as a DCOP Bob Bob E E M M Alice Alice 5 , 3 8 1 , 0 1 M M 2 0 , 2 2 , 4 6 E E Alice M E Alice Anna What if students can’t/won’t communicate preferences? Anna M E M E 8 1 2 6 Cost:

16. Standard model solutions • Easiest solution: Disclose preferences • An alternate approach: Add unary constraints • Problem: Can prove that this approach will fail on some instances

17. How its done these days The PEAV formulation: A1 A2 x1 x2 x21 x12 • Modified search space • Can’t be used with many local search algorithm! • Requires more space mirror variables hard constraints x2 x21 x12 x1

18. Introducing ADCOPs Different preferences on outcomes are not part of the standard model… Asymmetric constraints Formally: Captures the idea that each agent has a personal “table” with costs/gains of each outcome

19. ADCOPs • ADCOPs: • At least as expressive as existing model • Succinct representation • Used with existing local search algorithms Search can be improved by introducing cooperation/coordination

20. ADCOP Local Search (quality) DCOP ADCOP

21. Constraint Reasoning Distributed Computing Distributed Constraint Reasoning Multi Agent Systems

22. Rethinking agents joint objective Difference in best and worst gains – Meeting Scheduling Problem

23. Agreeing on an outcome(what is a fair solution?) • Game Theory defines stable points: • Assumptions: • Self interested • Rational (some knowledge)

24. Graphical Games Anna E M Alice 5 , 3 1 , 0 M • ADCOPs are Games played on a Graph • Closely related to Graphical Games • ADCOPs: • No knowledge assumed • Agents are cooperative • An even more succinct representation • Can use DCR techniques to solve a game theoreticmulti agent problem! 0 , 2 2 , 4 E

25. Asynchronous Nash BackTracking (ANT) • Transform a MAS to a Distributed Constraint Problem • A distributed, asynchronous, non-binary, asymmetric search Three asymmetric constraints Two symmetric constraints

26. ANT • A satisfaction problem • Inspired by ABT (ABT-1ph) • A solution always exists • Guaranteed to find an epsilon NE: • More efficient than other distributed GG solvers

27. Constraint Reasoning Distributed Computing Distributed Constraint Reasoning Multi Agent Systems

28. The quality of a stable solution • A stable solution is not necessarily a good one… • Why is that? • Competitive solution for cooperative agents?

29. Agreeing on an outcome(what is a fair solution?) Stable points: Nash (pure/mixed), Bayesian, Strong, Correlated, … Utilitarian, Egalitarian, Leximin,…

30. A different approach assume cooperation but try to incentivize agents by examining personal goals • “Cost of Cooperation” • Baseline search

31. The Cost of Cooperation (CoC) criteria: Non positive CoC solutions The difference in an agent’s gain from the worst equilibrium (from its point of view) outcome and from cooperatively solving the problem U2(x) Nash equilibrium solutions Pareto front Optimal solution (max sum) U1(x) Possible solutions

32. A simple P2P example u2=high C2 a2 a5 F S u1=low u1=med a1 a3 a6 a8 S S F F C1 a4 a7 • Agents only interact with neighbors(unknown topology) • An agent’s gain is lowered when exerting resources on sharing (S) • Gain is maximized if an agent can free ride the efforts of otheragents (F) • Gain is lowest if no one shares

33. Competitive and Cooperative solutions A Bayesian stable solution (possible) 0 1 1 0 a2 a5 a2 a5 F F F F 1 1 a1 a3 a6 a8 a1 a3 a6 a8 F F S F S S F S Cooperative Solution 1 1 0.3 0.3 0.3 0.3 a4 a7 a4 a7 F F F F 0 1 1 0

34. Cost of Cooperation solution • An improvement can be guaranteed (proved) for a set of interactions! 0.3 S 0 a2 a5 F a1 a3 a6 a8 F S F S S 0.3 1 1 0.3 0.3 a4 a7 F 0

35. Applied to network games Maximizing utilities ADCOP (CoC)

36. Constraint Reasoning Distributed Computing Distributed Constraint Reasoning Multi Agent Systems

37. Limits of the CoC approach • So far we have seen several solutions: • Fully cooperative (Utilitarian) • Stable (Epsilon Nash Equilibrium) • A combination: Non positive Cost of Cooperation • However… Mixed NE: (1/2,1/3) Gain: (3, 7/3) NO Cost of Cooperation solution!

38. A framework for partial cooperation • Agents gain is different • Do not “improve cooperatively” • Define cooperation with respect to some baseline solution • Agents must agree on the baseline (may need to apply a simple search algorithm).

39. Modes of cooperation • Define modes of cooperation within an Interaction Process: • Non-Cooperative (NC) – agents are driven by their own goals and act rationally. • Can serve as a baseline solution • Guaranteed Personal Benefit (GPB) – agents seek an agreement and may take irrational steps. • Guarantees a Pareto improvement • λ-cooperation – agents agree to a bounded loss from their NC gain (up to some predefined λ)

40. Local Search and Partial Cooperation Maintain threshold/guarantee: • Incorporate with distributed “anytime” • Can use any LS algorithm • Focus on exploration • Tailor an algorithm • maintain invariant (begins in a “legal” state)

41. Evaluation Three key parameters: • Compromise levels (lambda) • Agents’ degree • Costs distribution

42. Constraint Reasoning Distributed Computing Distributed Constraint Reasoning Multi Agent Systems

43. Summary & Conclusions

44. Summary DCSP/DCOP Representation Utilitarian(Minimal sum of costs) Multi Agent Problem Algorithm Objective Stableε-Nash Equilibrium Asymmetric Constraints Non positive Cost of Cooperation Partial Cooperation

45. Conclusions • Three points (‘up and down the ladder of abstraction’): • How to model the problem • How does the model effect the means to find a solution • What is a solution? • Rethinking basic assumptions • Applying well established models to simple realistic settings can reveal many of its shortcoming

46. Journal publications: • ArnonNetzer, Alon Grubshtein and AmnonMeisels, “Concurrent Forward Bounding”, Artificial Intelligence, Vol. 193, pp. 186-216, 2012. • RoieZivan, Alon Grubshtein and AmnonMeisels, “Hybrid Search for Dynamically changing CSPs”, Constraints, special issue on constraint satisfaction for planning and Scheduling, Vol. 16, num. 3, pp. 228-249, 2011. • Alon Grubshtein and AmnonMeisels, “Cost of Cooperation for Scheduling Meetings”, Journal of Computer Science and Information System (ComSIS), Vol. 7, num. 3, pp. 551-567, 2010. • Conference and workshops publications : • Alon Grubshtein and AmnonMeisels, “Finding a Nash Equilibrium by Asynchronous Backtracking”, 18th Intl. Conf. on Principles and Practice of Constraint Programming (CP’12), pp. 925-940, Quebec city, Canada, Oct. 2012. • Alon Grubshtein, RoieZivan and AmnonMeisels, “Partial Cooperation in Multi Agent Local Search”, 20th European Conf. on Artificial Intelligence,pp.378-383, Montpellier France, Aug. 2012 • RoieZivan, Alon Grubshtein, Michal Friedman and AmnonMeisels, “Partial Cooperation in Multi Agent Search”, (Extended Abstract) Proc. 11th intern. Conf. Autonom. Agents Multi agent Sys. (AAMAS’12), Valencia, Spain. • AlonGrubshtein and AmnonMeisels, “A Distributed Cooperative Approach for Optimizing a Family of Network Games”, Proc. of the 5th Intern. Symp. on Intelligent Distributed Computing (IDC’11), Delft, the Netherlands, pp. 49-62, October 2011. • Alon Grubshtein and AmnonMeisels, “A Distributed Cooperative Approach for Optimizing a Network Game”, Proc. 13th Intern. Workshop on Dist. Constraints Reasoning (DCR’11), Barcelona, Spain, June 2011. • Alon Grubshtein, NirHerschorn, ArnonNetzer, Guy Rapaport, Guy Yaffe and AmnonMeisels, “The Distributed Constraints (DisCo) Simulation Tool”, Proc. 13th Intern. Workshop on Dist. Constraints Reasoning (DCR’11), Barcelona, Spain, June 2011. • Alon Grubshtein and AmnonMeisels, “Cooperation Mechanism for a Network Game”, Proc. 3rd Intern. Conf. Agents and AI (ICAART’11), Rome, Italy, pp. 336-341, January 2011. • Alon Grubshtein, Tal Grinshpoun, AmnonMeisels and RoieZivan, “Local Search for Distributed Asymmetric Optimization”, Proc. 9th intern. Conf. Autonom. Agents Multi agent Sys. (AAMAS’10), Toronto, Canada, pp. 1015-1022, May 2010. • ArnonNetzer, AmnonMeisels and Alon Grubshtein, “Concurrent Forward Bounding for DCOPs”, Proc. 12th Intern. Workshop on Dist. Constraints Reasoning (DCR’10) at AAMAS’10, Toronto, May 2010. • Alon Grubshtein, Nurit Gal-Oz, Tal Grinshpoun, AmnonMeisels and RoieZivan, “Manipulating Recommendation Lists by Global Considerations”, Proc. 2nd Intern. Conf. Agents and AI (ICAART’10),pp. 135-142, Valencia, Spain, January 2010. • Alon Grubshtein and AmnonMeisels, “Cost of Cooperation for Scheduling Meetings”, Proc. 3rdIntern.Symp. Intell. Dist. Comp. (IDC’09), Vol. 237, pp. 227-236, Ayia Napa, Cyprus, October 2009. • Alon Grubshtein, Tal Grinshpoun, AmnonMeisels and RoieZivan, “Asymmetric Distributed Constraint Optimization”, Proc. 11th Intern. Workshop on Dist. Constraints Reasoning (DCR’09) at IJCAI-09, Pasadena CA, July 2009. • Ehud Gudes, Nurit Gal-Oz and Alon Grubshtein, “Methods for Computing Trust and Reputation While Preserving Privacy”, Proc. Data and App. Security XXIII, 23rd Ann. IFIP WG 11.3 Working Conf. (DBSEC’09), Vol. 5645, pp. 291-298, Montreal, Canada, July 2009. • Amir Gershman, Alon Grubshtein, AmnonMeisels and RoieZivan, “Scheduling Meetings by Agents”, Proc.7thintern. Conf. Practice and Theory Auto. Timetabling (PATAT’08), Montreal, August 2008. Thank you!