PLAN RECOGNITION & USER INTERFACES

1. PLAN RECOGNITION &USER INTERFACES Sony Jacob March 4th, 2005

2. AGENDA • Motivation • Examples • Introduction • Collaboration • Plan Recognition vs. Traditional Systems • Plan Recognition System “Steve” • Conclusions • Discussion

3. EXAMPLE: TRADITIONAL SYSTEM • Microsoft Interactive help system • Provides “jump-in” help instead of relevant on-going collaborative help • Unable to comprehend overall goals and does not use task model • Most users are unable to use this feature successfully

4. EXAMPLE: PLAN RECOGNITION SYSTEM Charles Rich Candy Sidner Neal Lesh Andrew Garland Shane Booth Markus Chimani 2004 Mistubishi Electric Research Laboratory

5. EXAMPLE: PLAN RECOGNITION SYSTEM Charles Rich Candy Sidner Neal Lesh Andrew Garland Shane Booth Markus Chimani 2004 Mistubishi Electric Research Laboratory

6. MOTIVATION • Minimize amount of initiative required from user • Create simple and consistent interfaces • Guide user without limiting capability

7. COLLABORATION DIAGRAM GOALS COMMUNICATION PRIMIVITIVE ACTIONS PRIMIVITIVE ACTIONS Model for Plan Recognition in user interfaces SHARED ARTIFACT (GUI)

8. COLLABORATION FRAMEWORK • Defined as an Interaction between “Agent” and User • Mutual Goals • Agent and user can perform actions • Agent uses Plan Tree • Hierarchal partially ordered representation of actions to achieve goals • Methods of interaction • Discussion between agent and user • User conveys intentions through actions • Agent solicits clarification

9. EXAMPLE: PLAN TREE Charles Rich Neal Lesh Andrew Garland 2002 Mistubishi Electric Research Laboratory

10. EXAMPLE: PLAN RECOGNIZER Charles Rich Candy Sidner Neal Lesh 1998 Mistubishi Electric Research Laboratory

11. RESPONSES • Possible actions for an agent • Move user to next step of task (goal oriented) • Confirm completion of a goal • Allow user initiative • Focus user to current goal • Explain steps needed for a task • Discover and report incorrect actions

12. RESPONSES CONTINUED • Traditional responses • Application dependent If(user pressed button A) Call function A • Collaborative responses • Application independent If(user completed a step in current task) Go to next step of task

13. INTERACTION • Traditional system • Limited range • Tutoring systems • Agent has majority of plan knowledge and initiative • Help system • User has majority of plan knowledge and initiative • Turn based interaction • User performs action and agent responds

14. INTERACTION CONTINUED • Collaborative System • Broad range • System can shift incrementally within this range • Examples mentioned in Traditional systems represent extremes of this range • Mode depends on current task • Non-turn based interaction • User may perform 0 or more actions followed by 1 or more communications • Agent may perform 0 or more actions followed by 1 or more communications

15. COMMUNICATION • Traditional Communication • User initiates system actions through “commands” • Requires user to have knowledge of command • Collaborative Communication • On-going Discourse between agent and user • Define goals and how to achieve them • Discuss task being performed • Requires user to have common goal with agent

16. HELP SYSTEMS • Traditional help systems • Wizards, Tool-tips, Help Assistants • Attempt to compensate for lack of user knowledge • Requires separate interaction by user • Collaborative help system • Integrated as part of the interface • User knowledge level does not affect level of help system interaction

17. EXAMPLE: WIZARDS • Wizards • Provide a guided interaction for user • Partially follows collaborative paradigm • Lacks versatility to allow user to take initiative • Goals cannot be adjusted

18. ADVANTAGES OF DIAMOND HELP • Provides consistent interaction paradigm • Different applications of Diamond help will be familiar to user • Appearance and operation remains the same • Used for appliances and control systems • Possible expansion allows for speech-enabled interaction • Agent speaks interaction and performs speech recognition for user

19. PLAN RECOGNITION SYSTEM EXAMPLE: STEVE • Training agent • Steve (Soar Training Expert for Virtual Environments) • Virtual reality tutoring system, agent embodiment • Uses plan recognition to guide user • Actions taken by user are interpreted and compared to plan tree • Steve orients user towards goal based on plan tree • Advises user when a deviation is made from the plan tree or when help is needed for the next step • Demo • http://www.isi.edu/isd/VET/steve-demo.html

20. CONCLUSIONS • Effective Collaboration • Abstract representation of situation • Key to reuse and modularity of components • Well-designed task model • Hierarchy must model tasks which complete a goal or sub-goal • Focus must be maintained • If goal is modified, focus “stack” must be adjusted

21. CONCLUSIONS CONTINUED • Complexity and scalability • Must be able to create abstract representations for various tasks • Depends on modularity and reusability of components • For complex interactions, must allow more direct user actions • Require Sub-goals for top goals

22. DISCUSSION • Questions? • How do we program a collaborative system? • What are the drawbacks of creating this system? • How does the agent realize a plan for a complex system? • How would I implement this in my workplace environment?

23. PROGRAMMING TECHNIQUE • Object Oriented Plug-ins • Use separate components to construct desired interface • Composable and reusable components • Abstract class definitions for dialogs • Low-level functionality is controlled and monitored by high-level plan recognition system

24. References • Note: Web addresses only • http://km.aifb.uni-karlsruhe.de/ws/LLWA/abis/schneider.pdf • http://www.merl.com/papers/docs/TR98-23.pdf • http://www.merl.com/reports/docs/TR2002-10.pdf • http://rpgoldman.real-time.com/papers/discex01pr.pdf • http://www.isi.edu/isd/VET/eca00.pdf