1 / 42

Toolkits for Physical Objects (TUIs)

Toolkits for Physical Objects (TUIs). Jack Li Advanced User Interface Software November 30 Fall 2004. Outline. Motivation for TUIs Input Technologies TUI Toolkits. MOTIVATION FOR TUIS. Current Limitations of UIs. Desktops and laptops suck in our entire attention

ura
Télécharger la présentation

Toolkits for Physical Objects (TUIs)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Toolkits for Physical Objects (TUIs) Jack Li Advanced User Interface Software November 30 Fall 2004

  2. Outline • Motivation for TUIs • Input Technologies • TUI Toolkits

  3. MOTIVATION FOR TUIS Current Limitations of UIs • Desktops and laptops suck in our entire attention • As a result we live in two worlds: the real physical world and the virtual cyber world • Constantly “wired” in order to be in both worlds • Loss of rich interactions found in everyday things (particularly haptic)

  4. MOTIVATION FOR TUIS Motivation • Driven by ubiquitous computing • Both people and technology moving away from the desktop into the environment • Input: different ways of interacting • Output: different ways of displaying

  5. MOTIVATION FOR TUIS Tangible User Interfaces • TUIs augment the physical world by integrating digital information with everyday physical objects • Goal: to bridge the gap between bits and atoms • 1. Interactive surfaces • 2. Coupling information with physical objects • 3. Ambient media for background awareness

  6. Outline • Motivation for TUIs • Input Technologies • TUI Toolkits

  7. INPUT TECHNOLOGIES Physical Objects as Input • Pretty much anything … walls, papers, utensils, clothes hangers! • Inspiration • sci-fi novels, own imagination • everyday life, current practices

  8. INPUT TECHNOLOGIES Devices • Cameras • Electronic tags (RFID technology) • Sensors • Light • Motion • Heat • Force • Motors • Orbs

  9. INPUT TECHNOLOGIES Framework for the GUI (a)model-view-controller

  10. INPUT TECHNOLOGIES Framework for the TUI (b)MCRpd: model-control-representation(physical and digital)

  11. Outline • Motivation for TUIs • Input Technologies • TUI Toolkits • Phidgets (2001) • Calder (2004) • Papier-Mâché (2004)

  12. Toolkit Goals • Same as when GUIs were built • Make easy for non-specialized experts to create (should not need a vision or hardware guru on the team) • Hide dirty implementation details • Abstract proper functionality with well-defined API

  13. PHIDGETS Overview • First toolkit for physical objects • Reusable physical devices with proper abstraction for creating a wide variety of applications • An API that had the right amount of abstraction

  14. PHIDGETS GUI Differences • Connection manager • Way of detecting when devices go on/off • Identification • Every physical object should have a virtual counterpart • Simulation • A way to test code even if phidget not present

  15. PHIDGETS Architecture

  16. PHIDGETS The Physical Device • Primitive device components • Sensors • Motors • Switches • Circuit board • CY7C63000 USB micro-controller • Communications layer • USB communication standard

  17. PHIDGETS Wire Protocol • Low-level software in Windows and microcontroller for communication that’s hidden from end programmers • Commands from phidgets • Unique Id numbers • Device type (servo, powerbar) • Specific events (tag for RFID reader) • Commands from Windows • Set device state (output like servo)

  18. PHIDGETS Computer Software • PhidgetManager • Connection manager • Event onAttach() • Event onDetach() • Int count • Enum item • Phidget-specific devices • All have: • String deviceType • Boolean isAttached • Long serialNumber • RFID • onTag() • Servo • onPositionChanged(), motorPosition(index)

  19. PHIDGETS Simple Application • Phidget Manager • Phidget Servo

  20. Outline • Motivation for TUIs • Input Technologies • TUI Toolkits • Phidgets (2001) • Calder (2004) • Papier-Mâché (2004)

  21. CALDER Main goals • Primarily geared towards interaction/product designers • Support for more refined products • Support on the hardware side

  22. CALDER Supported Devices Small size is big win!

  23. CALDER Fluid prototyping Objects easily embedded!

  24. CALDER Communication • Wired • PIC16C745 C with USB interface engine • Same as Phidgets (and iStuff) • Wireless • Tradeoff between size and power (perhaps Moore’s law will remedy) • Small batteries, antenna, and pulse-width modulation

  25. CALDER Software • Surrogates (GUI widgets) to manipulate physical object output

  26. Outline • Motivation for TUIs • Input Technologies • TUI Toolkits • Phidgets (2001) • Calder (2004) • Papier-Mâché (2004)

  27. PAPIER-MACHE Overview • Emphasis on software side side • Input model that provides common programming abstractions • Focus on objects being lightweight

  28. PAPIER-MACHE Software Abstractions • Phob—encapsulates information related to this physical object • All—source producer, time • RFIDPhob—tag • VisionPhob—mean color, bounding box, rotations • PhobListener—listens for Phobs produced • phobAdded(), phobUpdated(), phobRemoved() • PhobProducer—actual devices that generates events • PhobManager—maintains PhobProducers

  29. PAPIER-MACHE A Common Event Model • PhobListener is the interface among all devices • Methods are the same • phobAdded(), phobUpdated(), phobRemoved() • Vision • phobAdded() when new object detected, phobUpdated() when old object changed (size, color), phobRemoved() when old object disappears • RFID (different from what the paper says) • phobAdded() when tag detected, no phobUpdated(), phobRemoved() when tag disappears • Facilitates easier retargetting and substitutions (allows for more exploration and faster, prototyping)

  30. PAPIER-MACHE Marble Answering Example • 1. Get the manager that keeps track of all input devices (PhobProducers) RFIDManager manager = RFIDManager.getManager(); • 2. Set up a map associating all objects with an audio clip AssociationFactory factory = new AudioClipFactory(); AssociationTreeTable assocMap = new AssociationTreeTable(factory); • 3. Attach this map so new objects are assigned new clips and old ones get played manager.addPhobListenerForAllProducers(assocMap);

  31. PAPIER-MACHE A Behavior: Associations • A common behavior we noticed • New objects get stored/assigned, old objects get retrieved a database of sorts • An extension of PhobListener

  32. PAPIER-MACHE A Behavior: Associations • AssociationTreeTable • addClassifierFactoryPair(Classifier, Factory) • ObjectClassifier • isMemberOfClass(Phob phobToCompare) • AssociationFactory • createAssociationEltForPhob(Phob phob)

  33. PAPIER-MACHE Monitoring Window

  34. PAPIER-MACHE Device state—connection mgr

  35. PAPIER-MACHE Camera Feedback

  36. PAPIER-MACHE Association Map

  37. PAPIER-MACHE Extras

  38. Summary

  39. Major Toolkit differences • Phidgets • Complete, balanced, varied assortment of physical objects easy to program with • Calder • Small size allows for more fluid prototyping at the PHYSICAL level • Papier-Mâché • Unified event model allows easier development and retargetting at the SOFTWARE level

  40. Conclusion • TUIs seek to augment the physical world with digital information • Toolkit support allows non-experts to develop TUI application, like GUI toolkits • Successful toolkits must manage physical connections, gather/alter relevant physical state information, and provide good feedback for these added complexities

  41. Future work • Input adaptation • M to N mapping of input devices • Input models • What other kinds of behaviors can be abstracted • Will be more clear with the development of more applications

  42. References • Hiroshi Ishii , Brygg Ullmer, Tangible bits: towards seamless interfaces between people, bits and atoms, Proceedings of the SIGCHI conference on Human factors in computing systems, p.234-241, March 22-27, 1997, Atlanta, Georgia, United States • Ullmer, B. and H. Ishii, Emerging Frameworks for Tangible User Interfaces, in Human-Computer Interaction in the New Millennium, Addison-Wesley. pp. 579 -- 601, 2001. • Saul Greenberg, Chester Fitchett. "Phidgets: easy development of physical interfaces through physical widgets," Proceedings of the 14th annual ACM symposium on User interface software and technology, November 11-14, 2001, Orlando, Florida • Scott R. Klemmer, Jack Li, James Lin, James A. Landay. "Papier-Mache: toolkit support for tangible input," Proceedings of the 2004 conference on Human factors in computing systems, p.399-406, April 24-29, 2004, Vienna, Austria • Lee, J.C.; Avrahami, D.; Hudson, S.E.; Forlizzi, J.; Dietz, P.H.; Leigh, D.L., "The Calder Toolkit: Wired and Wireless Components for Rapidly Prototyping Interactive Devices", Designing Interactive Systems (DIS), ISBN: 1-58113-787-7, pp. 167-175, August 2004

More Related