1. Dance and Engineering in AR Augmented Reality Presented by: Michael Korostelev

2. CFL Lab the Computer Fusion Laboratory Director: Dr. Li Bai Exchange scholar Dr. Guohua Huang Students: James Ren Michael Korostelev Michael Huang Fengcheng Wu David Osinski

3. Project Showcase Project Showcase `` ````` `````````` Septa Electronic Payment Project 2004 – now ($150K + $1.0M) Wireless Distributed Operating System 2007 – now ($400K) Wireless Industrial Process Monitoring 2008 – now ($200K) Dance and Engineering in Augmented Reality 2010 – now ($50K) Sensing Technology for Import Food Inspection 2009 - now Wind-Solar Hybrid LED Lighting System 2010 - now

4. Art and Engineering Purely creative

5. Art and Engineering Entertainment

6. Art and Engineering Practical and profitable

7. Art and Engineering Educational

8. CFL Lab Artist Visions Film Festival • Visual Treasure Hunt • Visitors given clues about the town of Lambertville NJ • They explore the town to find QR codes • Scan QR codes and attach their own message • Visitors’ input becomes collaborative story

9. Art and Engineering These projects bring engineers close to artists and act as a bridge between the disciplines Each having a goal to relieve the apprehension laymen have to technology Develop tools for creative people, rather than develop creative works

10. Motivation Create a medium for rich creative works Inspire ideas from artists and engineers alike Enhance education and increase outreach Establish framework for fundamental research

11. Project Goal Enhance visual experience of a live performance with AR • Visually enhance storytelling • Engage and involve audience • Convey complex ideas • Reduce production costs • Unbind creative limitations

12. What Is AR ? Display Techniques Head Mounted Displays Handheld Display Spatial Displays

13. Dance and Engineering in Augmented Reality Camera/Projector Installation Augmented Stage • Marker less stationary objects • IR led marked moving objects • Volumetric objects • Dynamic lighting • Floor projection • High frame rate digital capture • High lumen projector with wide focal range • PC with accelerated graphics

14. 2D/3D Projector-Camera Calibration

15. 2D/3D Projector-Camera Calibration Homography(perspective transformation) Stereo Calibration

16. Kinect “Hacks”

17. RGBD Sensor Based Calibration • Calibrate RGBD sensor Depth and Image camera separated physically so images do not match Uncalibrated overlay Two different approaches to create image overlay - Homography - Stereo Calibration

18. Projector – Sensor Extrinsics Real Scene Projection Sensor Capture

19. Determining Projection Matrix • Determining the projection matrix

20. Determining Projection Matrix

21. Displaying the scene

22. Displaying the Scene