1 / 20

3D Mobile Internet Virtual Operator and 3D User Interaction

3D Mobile Internet Virtual Operator and 3D User Interaction . Lukas Kencl ( Director ) lukas.kencl@rdc.cz Jiří Danihelka (Project leader ) jiri.danihelka@rdc.cz Roman Hák (Expert developer) Ondřej Šindelář (Junior developer) Jan Kadlec (Junior developer)

hayden
Télécharger la présentation

3D Mobile Internet Virtual Operator and 3D User Interaction

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. 3D Mobile InternetVirtual Operator and 3D User Interaction Lukas Kencl (Director)lukas.kencl@rdc.czJiří Danihelka (Project leader)jiri.danihelka@rdc.cz Roman Hák (Expert developer) Ondřej Šindelář (Junior developer) Jan Kadlec (Junior developer) R&D Center for Mobile Applications (RDC) FEE, Czech Technical University, Prague 2009

  2. Project group:3D Mobile Internet Virtual Customer Care 3D Mobile Internet, Danihelka, Hák, Šindelář, Kadlec, Kencl

  3. Virtual Customer Care Centervirtual store of Vodafone • user can roam around the centre, browse catalogue and products, watch adverts, • avatars talk, provide assistance, offer and sell services and products • platform independent - VRML browser • desktop PC app, web browser or mobile phone • written Virtual Reality Makeup Language • VRML is similar to X3D • models were inspired by real Vodafone stores • customers can choose mobile phones, see popular advertisements and many other things • Link: http://internet3d.rdc.cz/vodafone/index.php • VRML browser is required (see next slide) 3D Mobile Internet, Danihelka, Hák, Šindelář, Kadlec, Kencl

  4. How to download a VRML browser • Download Cortona VRML browser from:http://www.parallelgraphics.com/products/downloads • Cortona3D Viewer 6.0 (recomended)works with Firefox and Internet Explorer • Cortona VRML Client 5.1works with Internet Explorer only • Pocket Cortonafor Windows Mobile devices 3D Mobile Internet, Danihelka, Hák, Šindelář, Kadlec, Kencl

  5. Virtual Customer Care Centeron mobile phones • uses Pocket Cortona VRML browser • only minor changes from desktop version(smaller textures) • some functionality is not available(no video-textures) • user still can move in 3D environment, interact with objects, • Commercialization: virtual customer-care platforms for 3rd parties (banks, shops) • Natural, multi-modal interaction – breaks tech barriers • Opportunity: novel means of access to Web resources 3D Mobile Internet, Danihelka, Hák, Šindelář, Kadlec, Kencl

  6. Project group:3D Mobile Internet Virtual RDC 3D Mobile Internet, Danihelka, Hák, Šindelář, Kadlec, Kencl

  7. Virtual RDC3D model of our research center on web • accurate and interactive model of RDC laboratory • uses the same technology asVirtual Customer Care project • visitors can see our GSM network devices, server room, seminar room and many other things • Link: http://internet3d.rdc.cz/VirtualRDC/index.en.html • VRML browser is required 3D Mobile Internet, Danihelka, Hák, Šindelář, Kadlec, Kencl

  8. Project group:3D Mobile Internet Foreground Extraction on Mobile 3D Mobile Internet, Danihelka, Hák, Šindelář, Kadlec, Kencl

  9. Object 3D-Modeling Input:Image Foreground Extraction on Mobile 9 3D Mobile Internet, Danihelka, Hák, Šindelář, Kadlec, Kencl • Rapid interactive and iterative image segmentation • Working implementation of GrabCut – state-of-the-art high-performance algorithm (Rother, Kolmogorov, Blake - Microsoft Research Cambridge - Grabcut: Interactive foreground extraction using iterated graph cuts. SIGGRAPH, 2004.) • Windows-Mobile based prototype implementation - feasibility • Essential for 3D object modeling input

  10. Project group:3D Mobile Internet 3D Mobile Talking Head 3D Mobile Internet, Danihelka, Hák, Šindelář, Kadlec, Kencl

  11. Talking Head • Speaking avatar - weather forecast, telling stories- voice synthesis, not interactive • Built on research results from the Department of Computer Graphics and Interaction (formerly CGG), CTU, in collaboration with IBM Research • RDC intensively cooperates with this group • Added platform independent architecture and mobile support 3D Mobile Internet, Danihelka, Hák, Šindelář, Kadlec, Kencl

  12. 3D Virtual Assistant Cross-platform Architecture GL Benchmark, HTC Touch Pro, Nov 2008 12 12 3D Mobile Internet, Danihelka, Hák, Šindelář, Kadlec, Kencl Presentation title in footer Presentation title in footer 07 June 2014 07 June 2014 • Goal: interactive automated 3D Virtual Assistant on mobile and fixed client • architecture: cross-platform solution • Qt library with graphic interface • rendering using OpenGL ES on top of Qt • interactive voice using SIP (PJSIP on mobile) • RT face rendering OK - current phone: up to 7000 triangles at 15 FPS with fog or point-light • demo Jan 2009: graphics feasibility prototype

  13. 3D Mobile InternetTalking with an Avatar on a Mobile Phone Avatar enhanced application on a mobile phone • platform independent application design • can run either on desktop PC or mobile phone Virtual worlds with talking avatar • Virtual Operator • Virtual Shop Assistant Speech animation • 3D graphics rendering • using MPEG4 head animation • able to render up to 9000 polygons model Interface for user interaction • voice interface for natural speech • classical button interface for noisy environment 3D Mobile Internet, Danihelka, Hák, Šindelář, Kadlec, Kencl

  14. Architecture of thesystem Client • our system uses client-server distributed architecture • application logic is stored on a remote server • server generates voice using Text-to-speech engine (TTS) • then for corresponding head animation Facial Animation Parameters (FAP) are generated in FAP Generator • stream of the voice and the FAP stream are put together in MPEG4 Encoder • mobile client decodes the stream and plays the voice and the animation Face (MPEG4-compillant) Geometry FAP stream Audio stream MPEG4-Decoder MPEG4Stream User action Server FAP stream Animation commands FAP Generator MPEG-4 Encoder Speech commands Audio stream TTS Engine Application server 3D Mobile Internet, Danihelka, Hák, Šindelář, Kadlec, Kencl

  15. Creation of head models • Tools for MPEG4 compliant heads • Set animation points on model • Important points:- center of the eyes- top of the nose- edges of the lips- ... • Points are used for meshchanges during animation • Uses only very small bandwidth 3D Mobile Internet, Danihelka, Hák, Šindelář, Kadlec, Kencl

  16. Project group:3D Mobile Internet Future plans 3D Mobile Internet, Danihelka, Hák, Šindelář, Kadlec, Kencl

  17. 3D Mobile Internet – further plans • SIP Client for Virtual Operator to Voice2Web platform Voice2Web SIP Client 3D Mobile Internet, Danihelka, Hák, Šindelář, Kadlec, Kencl

  18. 3D Mobile Internet – further plans Make interconnection of the projects Talking Head Automated Call Centernon-interactive (e.g. weather forecast) non-visual interface 3D Interactive virtual operator for mobiles and PCs 3D Mobile Internet, Danihelka, Hák, Šindelář, Kadlec, Kencl

  19. Conclusion R&D Center for Mobile Applications 3D Mobile Internet, Danihelka, Hák, Šindelář, Kadlec, Kencl

  20. Thank you! Do you see a synergy? For questions please contact: Dr. Lukas Kencl, Director R&D Centre (RDC) for Mobile Applications Czech Technical University Technicka 2 166 27 Prague 6 Czech Republic Tel.: +420-224355991 Fax: +420-224355999 lukas.kencl@rdc.cz See: http://www.rdc.cz Or better come in person? 3D Mobile Internet, Danihelka, Hák, Šindelář, Kadlec, Kencl

More Related