1 / 55

Direct Manipulation of User Interfaces for Migration

Intelligent User Interfaces 2006 Jan 29 - Feb 1 2005, Sydney, Australia. Direct Manipulation of User Interfaces for Migration. J. P. Molina 1,2 , J. Vanderdonckt 1 and P. González 2 1 Université catholique de Louvain 2 University of Castilla-La Mancha. www.usixml.org. Contents.

carson-silva
Télécharger la présentation

Direct Manipulation of User Interfaces for Migration

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Intelligent User Interfaces 2006 Jan 29 - Feb 1 2005, Sydney, Australia Direct Manipulation of User Interfaces for Migration J. P. Molina1,2, J. Vanderdonckt1 and P. González2 1 Université catholique de Louvain 2 University of Castilla-La Mancha www.usixml.org

  2. Contents • Introduction • MigriXML • Case study • Migration process in detail • Conclusion Intelligent User Interfaces 2006, Sydney, Australia

  3. Contents • Introduction • MigriXML • Case study • Migration process in detail • Conclusion Intelligent User Interfaces 2006, Sydney, Australia

  4. Introduction • End users of modern interactive systems face a wide variety of computing platforms to support their interactive tasks, from mobile phones to wall screens • The specific needs of the user’s environment at a given moment, or its evolution, can make necessary to remove one application from one platform and make it run in another one, that is, to migrate the application Intelligent User Interfaces 2006, Sydney, Australia

  5. Introduction • Migration can be: • Total, when the whole application is migrated • Partial, only the UI is migrated; if it is decomposed into control and presentation, then migration can be: • Control-oriented, migrates only the control component • Presentation-oriented, the control remains • Mixed, parts of both control and presentation are migrated • A user interface (UI) is said to be migratable if it holds the migration ability Intelligent User Interfaces 2006, Sydney, Australia

  6. Introduction • A special UI is required for controlling the run-time migration of the UI of the interactive systems • This UI is referred to as the meta-user interface, implemented in very different ways: • Sytem-initiated, the system initiates the migration • User-initiated, the user initiates the migration • Mixed-initiated, user and system collaborate to perform the migration Intelligent User Interfaces 2006, Sydney, Australia

  7. Introduction • Related work Intelligent User Interfaces 2006, Sydney, Australia

  8. Introduction • Elements represented in in the meta-UI Intelligent User Interfaces 2006, Sydney, Australia

  9. Introduction • Tasks are usually distributed among platforms, but without any consistent way to distribute them, the space between platforms introduces a discontinuity that may disrupt the fluency of the interactive task • Direct manipulation is proposed to tackle that discontinuity, and thus avoid possible usability problems Intelligent User Interfaces 2006, Sydney, Australia

  10. Contents • Introduction • MigriXML • Case study • Migration process in detail • Conclusion Intelligent User Interfaces 2006, Sydney, Australia

  11. MigriXML • In order to make user interface migration as natural as direct manipulation, a meta-UI in the form of a virtual environment has been developed • MigriXML enables the user to carry out that migration process without the physical presence of the involved platforms, in a virtual and interactive way, relaying on UsiXML language and a set of models Intelligent User Interfaces 2006, Sydney, Australia

  12. MigriXML • To generate the virtual reality scene representing the migration environment, a suite of models is used describing relevant aspects of the problem in terms of UsiXML: • The Concrete User Interface • The User Model • The Platform Model • The Environment Model • Describes properties of interest, which can be physical, psychological and organizational Intelligent User Interfaces 2006, Sydney, Australia

  13. MigriXML: Environment Model • The physical part consists of: • a scene model based on VRML97/X3D, • the multi-surface interaction ontology, • and the Stanford topology model. • Each scene is composed of physical resources and a series of planes, which are decomposed into areas • Each area may be an interaction surface: a monitor on a table, a wall screen, or any area where the UI is projected and/or recognized with computer vision • Each area is described by attributes like dimensions, relative position, relationships with other objects, etc Intelligent User Interfaces 2006, Sydney, Australia

  14. Interactive system architecture GUI application Arch meta-model or ‘Slinky’ meta-model Platform Layered structure MigriXML architecture Intelligent User Interfaces 2006, Sydney, Australia

  15. General MigriXML architecture User’s environment GUI application Arch meta-model fits the UsiXML models Platform Virtual Hardware (VHW) Virtual Machine (VM) Virtual Window Manager (VWM) Migration Manager (MM) MigriXML architecture Intelligent User Interfaces 2006, Sydney, Australia

  16. Implementation • This work focuses on the original part of providing a virtual control environment for migrating parts or whole of a UI from one platform to another • To address its implementation, the VRML97/X3D languages have been used • User Interfaces and the Arch meta-model: • Interaction toolkit component: VUIToolkit • Domain-specific component: Java and Javascript Intelligent User Interfaces 2006, Sydney, Australia

  17. Implementation • VUIToolkit • A set of VRML97/X3D PROTOs that transforms the standard plain widgets into a truly 3D representation (Virtualized User Interface) • This toolkit was developed starting from the object classes described in the Concrete Model of UsiXML language Intelligent User Interfaces 2006, Sydney, Australia

  18. Implementation • Java and Javascript • Most VRML97/X3D browsers can interpret Javascript code and execute Java code • VRML97/X3D specifications describe two programming interfaces to access the scene graph: • SAI (Scene Authoring Interface) • EAI (External Authoring Interface) • Allowed us to leave aside, for this first implementation of MigriXML, the complex integration of emulators for each specific platform Intelligent User Interfaces 2006, Sydney, Australia

  19. Interactive Java/Javascript application, virtualized user interface rendered in the 3D window of VRML97/X3D browser Implementation Intelligent User Interfaces 2006, Sydney, Australia

  20. Interactive application, virtualized user interface and virtual platform in MigriXML Implementation Intelligent User Interfaces 2006, Sydney, Australia

  21. Contents • Introduction • MigriXML • Case study • Migration process in detail • Conclusion Intelligent User Interfaces 2006, Sydney, Australia

  22. Case study • The user’s environment is a small office, with five different platforms Intelligent User Interfaces 2006, Sydney, Australia

  23. Case study • Display characteristics of each platform Intelligent User Interfaces 2006, Sydney, Australia

  24. Case study • UsiXML was used to describe the environment • The virtual world was the result of a structured process of creation: • Requirements: from UsiXML specifications, designers knew the details of the environment • Preparation: compilation of audio-visual material; sketches and mock-ups were created • Design, test and optimize: model creation, edit appearance, enhance world Intelligent User Interfaces 2006, Sydney, Australia

  25. Case study • User interfaces were specified in UsiXML too • Different tools have been developed to help the UI designer to visually create this models • IdealXML [Montero et al., 05] Intelligent User Interfaces 2006, Sydney, Australia

  26. Case study • User interfaces were specified in UsiXML too • Different tools have been developed to help the UI designer to visually create this models • IdealXML • GrafiXML [B. Michotte] Intelligent User Interfaces 2006, Sydney, Australia

  27. Case study • Two different user interfaces Intelligent User Interfaces 2006, Sydney, Australia

  28. 1 2 3 4 Case study • Migrating from one laptop to the other Aspire 200015”, 16:10 1280x800 PT-LB10SU4:3 800x600 Latitude15”, 4:3 1024x768 Intelligent User Interfaces 2006, Sydney, Australia

  29. 1 2 3 Case study • Migrating from a laptop to the Pocket PC Aspire 200015”, 16:10 1280x800 e7503,8”, 3:4240x320 Intelligent User Interfaces 2006, Sydney, Australia

  30. Contents • Introduction • MigriXML • Case study • Migration process in detail • Conclusion Intelligent User Interfaces 2006, Sydney, Australia

  31. Migration process in detail • The migration process is divided in four stages (A, B, C and D), each one having a finite set of steps, representing a total amount of 14 steps Intelligent User Interfaces 2006, Sydney, Australia

  32. Migration process in detail • Stage A: Select an interactive application Intelligent User Interfaces 2006, Sydney, Australia

  33. Migration process in detail • Stage A: Select an interactive application The user presses button (M) to grab the window, which follows the user wherever she points at Intelligent User Interfaces 2006, Sydney, Australia

  34. Migration process in detail • Stage A: Select an interactive application The action is transmitted to the corresponding Virtual Window Manager (VWM) by an event Intelligent User Interfaces 2006, Sydney, Australia

  35. Migration process in detail • Stage A: Select an interactive application The VWM forwards the information to the Migration Manager (MM) Intelligent User Interfaces 2006, Sydney, Australia

  36. Migration process in detail • Stage B: Select target platform Intelligent User Interfaces 2006, Sydney, Australia

  37. Migration process in detail • Stage B: Select target platform The MM broadcasts a message to all platforms Intelligent User Interfaces 2006, Sydney, Australia

  38. Migration process in detail • Stage B: Select target platform Platforms change their state to ‘wait-for-selection’, a TouchSensor is activated Intelligent User Interfaces 2006, Sydney, Australia

  39. Migration process in detail • Stage B: Select target platform The user selects the target platform by clicking on its screen Intelligent User Interfaces 2006, Sydney, Australia

  40. Migration process in detail • Stage B: Select target platform The Virtual Hardware (VHW) casts an event to inform the VWM Intelligent User Interfaces 2006, Sydney, Australia

  41. Migration process in detail • Stage B: Select target platform The VWM forwards the message to the MM Intelligent User Interfaces 2006, Sydney, Australia

  42. Migration process in detail • Stage C: Migrate to target platform Intelligent User Interfaces 2006, Sydney, Australia

  43. Migration process in detail • Stage C: Migrate to target platform The MM commands to detach the application from source platform Intelligent User Interfaces 2006, Sydney, Australia

  44. Migration process in detail • Stage C: Migrate to target platform 10 The MM broadcast amessage to the rest ofplatforms, which changeto normal execution Intelligent User Interfaces 2006, Sydney, Australia

  45. Migration process in detail • Stage C: Migrate to target platform The MM takes the UsiXML specification and re-generates the app UI Intelligent User Interfaces 2006, Sydney, Australia

  46. Migration process in detail • Stage D: Adapt to target platform Intelligent User Interfaces 2006, Sydney, Australia

  47. Migration process in detail • Stage D: Adapt to target platform The MM asks the target platform to host the re-generated application Intelligent User Interfaces 2006, Sydney, Australia

  48. Migration process in detail • Stage D: Adapt to target platform The application is launched, and its virtualized UI added to the scene graph of the target VHW Intelligent User Interfaces 2006, Sydney, Australia

  49. Migration process in detail • Stage D: Adapt to target platform The virtualized UI is adapted to the target screen and windowing system Intelligent User Interfaces 2006, Sydney, Australia

  50. Contents • Introduction • MigriXML • Case study • Migration process in detail • Conclusion Intelligent User Interfaces 2006, Sydney, Australia

More Related