1 / 22

Immersive Virtual Museum Experience with Kinect Interaction

Explore a virtual museum through natural user gestures with Kinect. Includes collision detection, object selection, and interactive tools. This project utilized VRJuggler for movement and bullet physics for realistic user interactions.

lonato
Télécharger la présentation

Immersive Virtual Museum Experience with Kinect Interaction

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Virtual Museum EugeniCasadesús, Ramón Oliva & Miguel Pasenau 2012, January 24th Virtual Reality, Immersive Interaction, Usability and Presence (RVA) course Master in Computing ( LSI – UPC)

  2. Contents • Goal • Implementation • Results • Conclusions

  3. Contents • Goal • Implementation • Results • Conclusions

  4. Goal • Virtual Museum: 3 rooms • User can move around: • Collision detection • Objects exposed, selectable for inspection

  5. Environment & tools • Cave or Power-Wall: • Limited space • User gestures to move around scene  Kinect • VRJuggler: • Targets Cave, Power-Wall and desktop computer as well • Kinect interface: • Natural user gestures • NITE + OpenNI + FAAST • Bullet: collision detection

  6. Contents • Goal • Implementation • Results • Conclusions

  7. Museum floor plan 13.964 triangles

  8. Objects 1.024 triangles 69.451 triangles 111.658 triangles 99.877 triangles 51.925 triangles 28.688 triangles 54.931 triangles 51.132 triangles 97.425 triangles

  9. User movements • Start & finish navigation: left arm up • Forward, backward, left, right: • Segway: lean forward / backward: moves fw / bw • Rotate torso left / right: rotates left / right • Left arm up / down: starts / ends movement • Selection • Wand ? • Stretching arm ? • Help • Raising hand ?

  10. VRJuggler approach • User stays fixed, scene moves • Init(): • Initialize devices • read ply models: walls, pedestals & objects • Create bullet physics: • Museum, objects and pedestals containers

  11. VRJuggler approach • preFrame(): • Update physics • Gets and process kinect and other events data • Draw(): • Navigation mode: draw museum and objects • Inspection mode: draw selected object zoomed

  12. Actual state • Museum, pedestals and objects are rendered • Collision detection is enabled • User movement is enabled: • Forward and backward, left and right turning • Starting Kinect integration: • Gestures recognition and application responds

  13. VRJuggler issues • No x64 alone build & install • Only mixed x32 and x64 Linux build and install • Using x32 build: • https://github.com/rpavlik/vrjuggler-windows-binaries • Units in feet and inches, not meters • User centred, scene moving • No restrictions or definition of movements • No event filtering possible

  14. Bullet + VRJuggler issues • User centred, scene moving • Translations  whole scene is translated the hard way • Rotations  whole scene is rotated the hard way • Collisions: • User: forward ray • Scene: bbox and cylinders • Object selection = collision wand ray - object

  15. Kinect issues • Unstable FAAST 0.10 • Using .jconf: lot of noise • Gesture recognition using FAAST, associate with keys / events of vrjuggler

  16. Contents • Goal • Implementation • Results • Conclusions

  17. Results

  18. Results • Source code at: http://code.google.com/p/museum-navigator/

  19. Contents • Goal • Implementation • Results • Conclusions

  20. Conclusions • Developing a RV application: • Hard to do • Still a juggling art • VRJuggler: • Hard to build and install: documentation so-so • Only x32 is known to work • Bullet: easy to install and to start with • Kinect: difficult to incorporate to VRJuggler: • Gesture analysis and interpretation • Complicated device and proxies structure

  21. References • Bullet documentation • VRJuggler documentation • FAAST documentation • http://www.keyboardmods.com/2010/12/howto-kinect-openninite-skeleton.html • https://github.com/avin2/SensorKinect • http://www.cs.unc.edu/Research/vrpn/

  22. Virtual Museum Eugeni, Ramón Oliva & Miguel Pasenau 2012, January 24th Virtual Reality, Immersive Interaction, Usability and Presence (RVA) course Master in Computing ( LSI – UPC)

More Related