850 likes | 938 Vues
Institut National de Recherche en Informatique et Automatique. Stanislaw Borkowski thesis director: James L. Crowley Jury:. Steerable Interfaces for Interactive Environments. Andreas Butz (UM), Jo ë lle Coutaz (UJF), Alex Pentland (MIT), Pierre Wellner (IDIAP). INRIA Rh ô ne-Alpes
E N D
Institut National de Recherche en Informatique et Automatique Stanislaw Borkowski thesis director: James L. Crowley Jury: Steerable Interfaces for Interactive Environments Andreas Butz (UM), Joëlle Coutaz (UJF), Alex Pentland (MIT), Pierre Wellner (IDIAP) INRIA Rhône-Alpes June 26, 2006
What is a user interface? • User Interface (UI): aggregate of physical entities or information bound to these entities A
Mobile UI’s • Steerable UI: • can be relocated in space • position is mediated by the computer system • Portable UI: • can be relocated • position is directly controlled through physical contact Mobile UIs Steerable UIs Portable UIs A
Mobility in current IT • Steerable interfaces • Conventional GUI (steerable output) • X11 session teleporting [Richardson93] • Portable interfaces • Wearable computers • Cell phones • Personal Digital Assistants • Laptops • ….
Mobility in ambient computing • Multiple displays embedded in the environment • Large size displays • Mobile interaction resources, both portable and steerable [Arias00] [Streitz99] [Pinhanez01]
Why steerable? • Flexibility in resources usage • New forms of Human-computer interaction • New forms of Human-Human interaction
Current situation – summary Problem: • Need for steerable UIs • No predictive models Solution: • Provide enabling technology • Explore interaction techniques • Evaluate the value of steerable UIs
Outline • Mobility in IT • Steerable UIs • Mobile projected UI • Mobile UIs for collaborative work • Conclusions
State of the art • EasyLiving [Brumitt00] • Tic-Tac-Toe [Pinhanez05]
State of the art – limitations • UI is observable at standstill • Limited spatial controllability • Only predefined locations • Planar surfaces only Requirements for steerable UIs: • Continuous observability and controllability
Outline • Mobility in IT • Steerable UIs • Mobile projected UI • Prototype implementation[in collaboration with J. Letessier] • Evaluation – latency estimation • Mobile UIs for collaborative work • Conclusions
The Steerable Camera Projector(2002) Other steerable projection systems: • The Everywhere Display (IBM 2000) • Fluid Beam (Fluidum consortium 2002) • SCP from Karlsruhe (UKA 2004)
User-centric approach End-users: • Latency limits < 50ms • Easy setup, no maintenance • Reliability / predictability Developers: • Abstraction: be relevant • Isolation: allow integration • Contract: offer quality of service
Pragmatic approach • Black-box services • BIP(Basic Interconnection Protocol) • BIP implementation ≈ SOA middleware • service/service and service/application communication • service discovery (standards-based)
Interactive system SCP software Human and Environment Interaction events Application Display orders
Human and Environment Interactive system Interaction detector Frame grabber Application SCPdisplay SCPcalibrator SCPcontroller A
Human and Environment Interactive system Interaction detector Frame grabber Application SCPdisplay SCPcalibrator SCPcontroller
Screen Video projector Source image Projection on arbitrary oriented planar surfaces Light source User’s perception
Screen Video projector Image to project Source image Projection on arbitrary oriented planar surfaces Light source SCPdisplay User’s perception
Projection on arbitrary oriented planar surfaces User’s view Image to project
Human and Environment Interactive system Interaction detector Frame grabber Application SCPdisplay SCPcalibrator SCPcontroller
3 Y 2 1 Q Sensor-centric environment model 2 3 1
Display surface detection Screen
Human and Environment Interactive system Interaction detector Frame grabber Application SCPdisplay SCPcalibrator SCPcontroller
Touch detection • Locate widget in the camera image • Estimate occlusion • Update widget state
Striplet-based SPOD SPOD – Simple-Pattern Occlusion Detector
SPOD-based calculator Accelerated video
Outline • Mobility in IT • Steerable UIs • Mobile projected UI • Prototype implementation • Evaluation – latency estimation • Mobile UIs for collaborative work • Conclusions
PCI A/D converter Frame Grabber Graphic Card OpenGl render Latency estimation CPU Imalab shell Image processing PDS
PCI A/D converter Frame Grabber Graphic Card OpenGl render Latency estimation Video sequence capture CPU Imalab shell Projection of the bar Image processing Plastic bar Fan Regulated power supply
PCI A/D converter PCI A/D converter Frame Grabber Frame Grabber ~17ms CPU Imalab shell Image processing ~70ms Graphic Card OpenGl render CPU Imalab shell ~32ms Graphic Card Latency estimation – results A + up to 51ms!!!
Human and Environment Interactive system Interaction detector Frame grabber Application SCPdisplay SCPcalibrator SCPcontroller
Outline • Mobility in IT • Steerable UIs • Mobile projected UIs • Mobile UIs for collaborative work • ContAct application • User study – comparison of different take-over techniques • Conclusions
ContAct – a system for authoring presentations Collaboration through interface mobility
ContAct application setup • Wide angle camera • Tabletop camera • Steerable Camera Projector • Portable Display Surface
Outline • Mobility in IT • Steerable interface prototype • Mobile UIs for collaborative work • ContAct application • Taking control: a comparative user study [in collaboration with J. Maisonnasse and J. Letessier] • Conclusions
Evaluation of techniques for taking control Objectives: • Determine the preferred control taking technique • Evaluate the impact on the task completion performance • Evaluate user acceptance of steerable interfaces
Experimental setup Hardware: • Steerable Camera Projector • Microphone headsets • Portable Display Surface GUI Software: • Speech detector [D. Vaufreydaz] • Conversation modeling [J. Maisonnaisse] • Finger tracking [J. Letessier] • PDS tracking • Drawing application Users
The task Collaborative reconstruction of a graph
The task Collaborative reconstruction of a graph User 1 User 3 User 2