Download
huddle and information triggers helping users automate complex and temporally distant tasks n.
Skip this Video
Loading SlideShow in 5 Seconds..
Huddle and Information Triggers: Helping Users Automate Complex and Temporally-Distant Tasks PowerPoint Presentation
Download Presentation
Huddle and Information Triggers: Helping Users Automate Complex and Temporally-Distant Tasks

Huddle and Information Triggers: Helping Users Automate Complex and Temporally-Distant Tasks

315 Views Download Presentation
Download Presentation

Huddle and Information Triggers: Helping Users Automate Complex and Temporally-Distant Tasks

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. Huddle and Information Triggers:Helping Users Automate Complex and Temporally-Distant Tasks Jeffrey Nichols IBM Almaden Research Center UC Berkeley June 14, 2007

  2. Today – Two Mini Talks • Huddle • Automatic generation of interfaces for systems of connected appliances • Part of my dissertation work on the Personal Universal Controller • Information Triggers • Helps users recall important information or tasks at the moment they are important • New project at IBM Almaden Research Center • In collaboration with Jake Wobbrock at the University of Washington

  3. The Multi-Appliance Problem

  4. The Multi-Appliance Problem, cont. How can we provide improved interfaces for systems of connected appliances?

  5. Control of Multiple Appliances Universal Remote Controls Built-in Interfaces Custom Installations

  6. Huddle • Improved multi-appliance interfaces based on content flow Flow-Based Interface Aggregate User Interfaces

  7. Personal Universal Controller (PUC) • Automatically generates interfaces for remotely controlling appliances [Nichols, UIST 2002] • Specification language for describing appliance functionality Dependency information that describes the availability of functions based on appliance state • Automatically generate new interfaces to be consistent with previous interfaces [Nichols, CHI 2006] Knowledge base that identifies and stores similarities between appliances

  8. Related Work • Roadie [Lieberman, IUI 2006] Goal-oriented natural language interface for appliances Uses planner to configure appliances based on common sense database (EventNet) • ICrafter [Ponnekanti, Ubicomp 2002] Builds aggregate user interfaces based on “service interfaces” Hand-coded interface aggregators Limited automatic generation capabilities • Omojokun, et al. [Omojokun 2006] Examined how remote control interfaces might be generated from recordings of actual usage Recorded remote control usage with IR receiver • General Work on Interface Generation • Task-based work: TERESA [Mori 2004], XIML [Puerta, IUI 2002], others… • SUPPLE[Gajos, IUI 2004]

  9. Huddle Overview • Introduction • Content Flow Modeling • Flow-Based Interface • Aggregate User Interfaces • Discussion and Future Work

  10. Content Flow Model • Home Theater DVD Receiver BroadcastTelevision TV VCR #1 VCR #2

  11. Modeling Tasks • Home Theater: Watching Television DVD Receiver BroadcastTelevision TV VCR #1 VCR #2

  12. Modeling Tasks • Home Theater: Watching Television DVD Receiver BroadcastTelevision TV VCR #1 VCR #2

  13. Modeling Tasks • Home Theater: Watching Television DVD Receiver BroadcastTelevision TV VCR #1 VCR #2

  14. Modeling Tasks • Home Theater: Watch DVD DVD Receiver BroadcastTelevision TV VCR #1 VCR #2

  15. Modeling Tasks • Home Theater: Watch DVD + Copy Tape DVD Receiver BroadcastTelevision TV VCR #1 VCR #2

  16. Modeling Tasks • Presentation Room VCR PowerPoint Projector Laptop Task Manager External Video Control Media Player

  17. Modeling Tasks • Presentation Room: Presenting PowerPoint VCR PowerPoint Projector Laptop Task Manager External Video Control Media Player

  18. Where does the model come from? DVD Receiver BroadcastTelevision TV VCR #1 VCR #2

  19. Where does the model come from? DVD Receiver TV VCR #1 VCR #2

  20. Where does the model come from? DVD Receiver TV VCR #1 VCR #2

  21. Flow-Based Interface (FBI) Flow-Based Interface • Allows user to manage high-level tasks • Quick specification of flows • AI Planner automatically configures appliances • Question/Answer interface to address conflicts

  22. Flow-Based Interface

  23. Flow-Based Interface

  24. Flow-Based Interface

  25. Flow-Based Interface

  26. Flow-Based Interface

  27. Flow-Based Interface

  28. Flow-Based Interface

  29. Flow-Based Interface

  30. Flow-Based Interface

  31. Aggregate User Interfaces (AUIs) Aggregate Controls for “Watch DVD” • Allows user to manage low-level details • Control or setup of active flows • General setup of all appliances • Merged functions

  32. Aggregate User Interfaces • Three kinds • Control of Active Flows DVD Receiver BroadcastTelevision TV VCR #2 VCR #1

  33. Aggregate User Interfaces • Three kinds • Control of Active Flows DVD Receiver BroadcastTelevision TV VCR #2 VCR #1

  34. Aggregate User Interfaces • Three kinds • Control of Active Flows DVD Receiver BroadcastTelevision TV VCR #2 VCR #1

  35. Aggregate User Interfaces • Three kinds • Control of Active Flows DVD Receiver BroadcastTelevision TV VCR #2 VCR #1

  36. Aggregate User Interfaces • Three kinds • Control of Active Flows DVD Receiver BroadcastTelevision TV VCR #2 VCR #1

  37. Aggregate User Interfaces Volume for TV Volume for Receiver • Three kinds • Control of Active Flows “Copy Tape” “Watch TV using TV speakers” “Watch TV using Receiver speakers”

  38. Aggregate User Interfaces • Three kinds • Control of Active Flows • Setup of Active Flows Two screens of “Watch DVD” setup

  39. Aggregate User Interfaces • Three kinds • Control of Active Flows • Setup of Active Flows • General Setup of all appliances Two screens of General Setup for a home theater config.

  40. Aggregate User Interfaces • Three kinds • Control of Active Flows • Setup of Active Flows • General Setup of all appliances • Merged Functions • Most functions shouldn’t be merged • Some should: clock, language, sleep timer… • Future work… Baudisch’s flat volume control General merging Some merged functions

  41. Discussion • How can our use of content flow be extended? • Lighting in a presentation room • Video-conferencing rooms • Outside the home: factories & offices • Kitchen?

  42. Future Work • Extending to more detailed content types • Home theater: support for HD signals, encoded audio, etc. • Setup and Troubleshooting • Tool for specifying initial wiring • Helping with Problems • More aggregate interfaces • Usage-based aggregate? • Usability evaluations

  43. Information Triggers

  44. Information Triggers • Motivation It’s easy to find stuff on the web It’s even easy to re-find something you’ve previously found... ...if you remember to re-find it when you need it “Participants [in a re-finding study] complained that they often forgot to look for an information item until the period of its usefulness had passed.” (Jones et al. 2005)

  45. Information Triggers • What we’ve built so far • A Firefox extension that allows people to associate web pages with future events at which that page will be particularly relevant • When the event occurs, the web page is automatically opened for the user • Goal is to make triggers as easy and lightweight to configure as possible • Supports three different types of triggers: • Date & time • Arrival of an e-mail • Visit to a web page

  46. Beyond Current Reminder Systems • Existing reminder systems provide a short message about their topic, but info triggers provides the information itself • Context-based reminder systems (e.g. Cybreminder) have mainly used location for context

  47. Scenarios • Waiting to login to a website to configure account information (date & time trigger)

  48. Scenarios • Resuming a task that can’t be completed in one sitting, e.g. waiting for manager approval (new e-mail trigger)

  49. Scenarios • Opening a note-taking site when visiting a page with specific keywords (page visit trigger) information retrieval web pages

  50. Scenarios • Delaying a task until a leisure period (page visit trigger)