Modeling Agents’ Reasoning in Strategic Situations

1. Modeling Agents’ Reasoning in Strategic Situations Avi Pfeffer Sevan Ficici Kobi Gal

2. The Big Question • Why do agents (people or computers) do things in strategic situations? • Possible directions • social norms • cognitive or computational limitations • beliefs • preferences • reasoning patterns

3. Outline • Characterizing the reasoning patterns agents use (Avi) • The Colored Trails framework (Kobi) • Modeling people’s preferences and reasoning in dynamic situations (Kobi) • Modeling how people reason about other people (Sevan) • Modeling the subjective beliefs people have about other people (Sevan)

4. Why Understand Reasoning Patterns? • Understand people’s strategies: people are likely to prefer some reasoning patterns to others • Explanation: if we want to explain people’s strategies, or we want to explain computational agents’ strategies to people, it is useful to know what reasoning patterns are being employed • Computational benefits • Analogy to logic

5. Multi-Agent Influence Diagrams (MAIDs) • Graphical representation of strategic situations, based on Bayesian networks • Compact representation • exploits independences in decision making • naturally captures limited observations • capture structure in multi-attribute utility functions • Tell the “story” of a game • Ideal representation for thinking about reasoning patterns

6. Seismic Structure Test Test Result Oil Drill Profit Bayesian networks • Represent probability distribution • Nodes are random variables • Edges represent direct influence • Each node has conditional probability of the node given its parents P(Oil | Seismic Structure)

7. Seismic Structure Test Test Result Oil Drill Profit Influence Diagrams • Represent single-agent decision problems • Chance nodes (ellipses) • Decision nodes (rectangles) • Utility nodes (diamonds) • Parents of decision node represent information available to decision maker at time of making decision

8. Multi-Agent Influence Diagrams Seismic Structure Test • Represent multi-agent strategic situations • Like influence diagrams, except that decision and utility nodes are associated with specific agents Test Result Oil Drill Tester’s Profit Driller’s Profit

9. Reasoning Pattern #1: Direct Effect • An agent takes a decision because of its direct effect on its utility • without being mediated by other agents’ actions DA UA

10. Influencing • All other reasoning patterns fall under the category of influencing: trying to get another agent to do something that is beneficial to you • The possible reasoning patterns depend on what strategies are considered for other agents • Let’s restrict attention to those strategies where the other agent has “good reason” to play them

11. Reasoning Pattern #2: Manipulation • B knows about A’s action • A cares about B’s action • A’s action influences B’s outcome, so B has to react to what A does • A can manipulate B to respond to her in a favorable way DA DB UA UB

12. Reasoning Pattern #3: Signaling C • A communicates something that she knows to B, thus changing B’s behavior • Interesting point: A must care about the thing she is communicating, otherwise B won’t believe her DA DB UA UB

13. Reasoning Pattern #4: Revealing/Denying DA C • A causes B to find out about information A herself does not know • Tiger example • Also works the other way round E DB UA UB

14. Key Question • Are these all the reasoning patterns? • Answer: it depends what strategies you allow for other agents • If you allow general strategies, any pattern in which there is a directed path from a decision node to a utility node is a pattern • But if we restrict attention to a more restricted class of strategies, we get a more nuanced answer

15. Well-Distinguishing Strategies • Intuition: if a strategy makes a distinction, the distinction should make a difference • A well-distinguishing (WD) strategy is one in which • if the strategy distinguishes between two values of the parents of a node, the expected utility is different for the two values • if the strategy assigns different probability to two actions, the expected utility of the two actions is different

16. Reassuring Fact • Theorem: The set of WD strategies always includes a Nash equilibrium • But not all Nash equilibria are WD • And not all WD strategies are Nash equilibria

17. Completeness Result • Theorem: If other agents are playing WD strategies, then the four patterns of reasoning described earlier are the only patterns in which an agent cares about its decision

18. Converse • When one of the reasoning patterns holds, does an agent necessarily care about its decision? • Answer: not in general • But we can prove a weak converse: • Theorem: If one of the reasoning patterns holds in a MAID, there exist parameter values for the MAID such that the agent cares about its decision • Similar situation to Bayesian networks

19. Opportunities for Synergistic Research • What reasoning patterns do people use? • Do people prefer some reasoning patterns to others? • How can we study this in the lab or the field? • What about non-WD strategies (which I think people use) – can we find a good solution concept that adequately capture’s people’s behavior?