Breakout Group: Understanding

1. Breakout Group:Understanding Virginia Dignum Stefan Kopp Rianne van Lambalgen Richard Stocker Gita Sukthankar

2. Overview • Why we need understanding • Who needs to understand what (at what time) • Everyone needs to understand according to their own abilities • Don’t need to understand everything if entities have an ability to ask for help or ask for interventions • How do you measure understanding Understanding is a dynamic co-constructed process rather a passive one-way set of observations.

3. Grand Vision Representation language Shared explicit team, plans, and goals Humans Natural language Vision Haptics, gesture recognition Interface AgentsRobots Mutual predictability Understanding Key technologies: Intention recognition Explainable AI Learning by demo Commonsense reasoning Knowledge Common- Sense Assumptions Actions: Communication Learning Reduce uncertainty

4. Findings • Humans, agents, and robots have different capabilities that should be leveraged to improve understanding across all team members. • Issues of ethics and responsibility: if human takes primary responsibility they must have adequate understanding • Trust and understanding: trust is important for engendering understanding • Organizational structure dictates requirements. • Blackbox understanding: try to make boxes more transparent vs. understanding content of box

6. Understanding • How do you make sure that the team partners understand each other • Intention recognition • What understanding has to do with teamwork—how can it help • Theory of mind • What do we mean by understanding each other--definitions • Mutual predictability—humans need to predict what the robots are going to do

7. Challenges • You can never understand everything but humans seem to form teams anyways. • You can understand what is going on within the shared plan • Supporting human-robot teamwork rather than taskwork • Partial observable/unexpected events • Communication to remove ambiguity • Irrecoverable actions • Must be able to query the capabilities in an open source system

8. Challenges • Explainable AI: need to be able to explain why the agent is doing something • Commonsense understanding • Sensor uncertainty • Cup on the table to prevent papers from blowing away---difficult for robots to understand what is going in • Previously-cached plans • People don’t understand Roombas • You need to know what the robot is planning to do so you can take action

9. Understanding • Want your interactions to be pleasurable, not confusing • Difficult to program NPCs and opponents in game worlds even with perfect sensors (elevator riding) • Interleaving plans • Shared mental models • Developmental science • Learning by demonstration

10. Understanding • Assume different things from agents, robots, and humans • Don’t assume that robots have ulterior motives • People are more flexible, multi-tasking with many agendas • Robot vacuum cleaner, not spying for mother-in-land • Can we trust agents and robots? • Trust ATMs over human tellers in cash counting • Machines are assumed to do the right thing • Blackboxes: not everything can be investigated

11. Understanding • Villages/cannibals problem: need to phrase things such that the answer is unambiguous • Difficult to understand things in real-time • Domain specific: medical robots vs. household robots • How to move into a non-sandbox environment • The ability to ask people for plan • Create a team where people do things that they are good at • Knowing your own limitations in understanding • Knowing what you don’t know, understand self and limitations

12. Issues • Robots and humans don’t have the same expertise • Robots need to know that they can’t perform common-sense reasoning • Dirty, dull, and dangerous • Complicated • Need humans to take responsibility • Robot telepresence

13. Issues • Do we need understanding? • Understanding and common-sense • People/agents/robots are blackboxes---open the blackboxes and understand what is understanding • What are milestones that are going to get us to understanding • Sandboxes moving to more complicated situations

14. General Diagram • HCI---haptics, gesture recognition • Natural language • Computer vision • “perfect communication” • Understanding • Humanagent, robots • Knowledge, plans and goals • Common sense • Shared explicit team, model • Agents and robots can cooperate to know what the human is doing

15. Explicit model of the goals of the teams • Need to have a large enough database of goals and plans • What if the goals change (dynamic) • Understanding should be actively constructed • ActuibsL ommunication, social actions, moving robot • Learning or adaptation • Common sense, cultural database

16. Not possible to get full understanding---not worse in a human-agent-robot team • Shared understanding • Mutual predictability • Making robots behave randomly increases humans belief that the robot is behaving intelligently • Assumptions help us create understanding (closed-world) • Level of autonomy that the team members • Don’t want to have to solve the grand AI problem

17. Negotiate with your robot to get it to clean where you want