Augmented Reality and Visualization

1. Augmented Reality and Visualization Lecture 29 By Prof. Dr. Sajjad Mohsin

2. Today we shall cover • Augmented Reality • History, Applications, Limitations, Future • Visualization • Definitions • History • Types • Domains of Applications

3. What is Augmented Reality? • Augmented reality is a combination of a real life scene from a person, and a virtual scene from a computer that augments the scene with additional information • Specifically, augmented reality superimposes audio, graphics and other virtual affects upon the surrounding environment.

4. The Goal • Augmented reality enhances one's perception. The real world is simply the foundation. The goal is to make a system that will be so efficient that the user will not be able to tell the difference between the real world and the virtual augmentation.

5. Virtual vs. Augmented • Virtual Reality is defined as, "a computer generated, interactive, three-dimensional environment in which a person is immersed.” • In other words, virtual reality completely immerses the use in the virtual environment • Augmented reality virtually changes the real world while allowing the user to keep a sense of presence about his surroundings

6. Performance Criteria • The mechanism that augments reality can be judged based on two things • Update rate for generating the augmented image • Accuracy of the registration of the real and virtual image So in order for AR to be perceived as “real,” the rate at which the system renders the image must be no fewer than 10 images a second. It also means that the system must be able to register its environment accurately enough so that a virtual change can be super imposed

7. The Hardware • In order for the AR system to work, certain hardware is needed, including: display, tracking, input devices, sensors and a processor. • Currently, components such as a CPU, camera, display, accelerometer, GPS and solid state compass.

8. Display • One can perceive augmented reality in generally three different ways: • Head mounted: the most immersive experience, HMD puts the virtual scene over the user’s view of the world. The device is either optical see-through or video see-through • Handheld: the most commercially successful AR system, handheld display is generally video see-through with a gyroscope in order to orient the image • Spatial: any AR device separated from the user. Can be used by multiple people. In some cases, users are able to touch virtual physical objects giving a sensation of realness.

9. Software • AR vision methods are based on visual odometry, or determining position using camera images • This generally includes feature detection and other image processing methods • It also requires a real world coordinate system to be made once the image is processed, created by perhaps fiduciary marks on the image, simultaneous localization and mapping of the image, or other mapping methods

10. Applications: Medical • Augmented Reality has the ability to enhance the efficiency of surgeries and diagnostics • Virtual x-rays • Real time ultra-sound images • Nuclear magnetic resonance images • MRI image

11. Applications: Military • Numerous applications: • Detect objects not discernable to the naked eye, such as toxins • Examine sound of gunfire to determine location of shooter and type of weapon • Enhanced visual feedback of surroundings • Represent physical objects that aren’t yet visible, such as an airplane that has yet to land • Displays in cockpits of military vehicles • War game simulation technology • Etc.

12. Applications: Navigation • Building navigation; in order to analyze and explore buildings effectively • Military operations • Disaster management • Heads up displays that cover the displays of automobiles, planes, etc. • Individual navigation through new urban areas (street to street, place to place (GPS))

13. Applications: Entertainment, Performance and Education • Augmented Reality systems can create virtual objects and environments that enhance the experience of entertainment, and the quality of education • Musical performances can be augmented to add to the experience or create certain sound effects • Interaction with educational tools • Virtual guides that can even activate based on location on GPS or proximity to marker

14. Applications: Digital Applications • Since smart phones come with most of the basic components for an AR system, applications soon began to flood the smart phone market, making AR commercially successful.

15. Smart Phone App -- Wikitude • Gives the user data about their surroundings by overlaying information on the real-time scene taken by the phone’s camera

16. Smart Phone App -- Augmented ID  • Gives the user the ability to identify someone by looking at them through the camera of the phone

17. Smart Phone App -- Image Space • Lets the user record messages, photos and video and tag them with both place and time. Then, when someone else goes to that same location they can see the images that were left in the area

18. Smart Phone App -- Layer • Positioning the camera over your environment, the phone can overlay images from web pages and information about businesses in the area

19. Smart Phone App -- Video Games • A game is generally overlayed on to the user’s surroundings, then the user may interact with the scene through the phone

20. Everyday Augmented Reality • Powers immersive video games • Very common to use it to navigate one’s surroundings • Interact and understand digital content more interestingly • Clear benefits recognized in using AR technology • Now found on most smart phones, opening up a whole new way to interact with the digital and physical environment • There is now an entire market for AR apps, including one’s that possibly give schematics for a building, the constellations in the sky or the information about nearby restaurants • AR now available in Adobe Flash, making AR accessible through web browsing • Recently there has been a new form of crude 2D bar code technology, which can be used to quickly transfer information

21. Limitations and Cons • Does not work as well indoors • GPS only accurate within 9 meters • Danger of ‘too much information,” where a disconnect forms between the user and the real world • Privacy will be harder to keep due to virtual interaction

22. Limitations and Cons • Over reliance on AR technology • Image recognition technology not developed to potential yet • Inaccurate depth perception • Calibration and latency still poor

23. The Future of Augmented Reality • Next generation of smart phones will be able to handle augmented reality to a much greater extent due to improved batteries, compasses, graphics chips and processors. • The mobility of the smart phone and the location based services make AR a very popular investment.

24. Future…. • The line between virtual and physical begins to blur as augmented technology grows • Greater access to information based on AR will mean greater awareness of surrounding • Virtual graffiti and art could be developed • Desire for increased AR technology in the field of medicine, as well as other fields.

25. Information and Data Visualization

26. Visualization • “A picture is worth more than a thousand words”. – a Chinese proverb • “A picture is worth more than a thousand numbers”.

27. It looks like a swirl. There are smaller swirls at the edges. It has different shades of red at the outside, and is mostly green at the inside. The smaller swirls have purple highlights. The green has also different shades. Each small swirl is composed of even smaller ones. The swirls go clockwise. Inside the object, there are also red highlights. Those have different shades of red also. The green shades vary in a fan, while the purple ones are more uni-color. The green shades get darker towards the outside of the fan......

28. Terrain geometry: (10,20,21), (12,13,14), (13,32,12),...., (1,2,3), (2,4,5),(3,5,6),..... Terrain Texture: (23,34,54), (23,34,23), (45,26,78),.... Time 0: Volumetric cloud cover: 0, 0, 12, 14, 15, 15, 17, 12, 23, 45,..... Wind vectors: (0.2, 0.3, 0.93,5), (0.4,0.5,0.76,12),..., Time 1: Volumetric cloud cover: 0, 0, 11, 12, 13, 16, 20, 12, 32, 45,..... Wind vectors: (0.4,0.5,0.76,12),(0.5,0.5,0.7,6),...

29. What Is Visualization? • “seeing is believing” • we observe and draw conclusions • seeing is also understanding • beware of ‘illusions’ (magicians)

30. What Is Visualization? • Transformation of data or information into pictures • engages primary human sensory apparatus - vision

31. What Is Visualization? • Is a Tool for: • Aid For Learning/Understanding • Compact Representation Of Information (e.g. Numbers) • “Carrier” of Information

32. Visualization Flavors? • Scientific Vis. - User Interfaces, data representation/processing Algorithms, Visual Representations • Data Visualization - Include financial data and statistical methods • Information Visualization - Abstract Data: WWW documents, file structures, arbitrary relationships

33. History • 1137 - earliest known map (China) • 1603 - first star charts by Johann Beyer • 1637 - cartesian coordinate system (Descartes)

34. History (2) - Statistical • 1686 - first meteorological chart (Halley) • 1693 - mortality tables of city of Breslau (Halley) -> first attempt to correlate two variables

35. History (3) - 2D • Approx. 1750 - contour lines (height) • 1817 - isotherms (temperature) • 1829 - isochromatic lines (color) • 1864 - isobars (pressure)

36. History (4) - 3D Imaging • 1895 - X rays by W. Röntgen • 1898 - stereo X rays (mackenzie-davidson) - locating foreign bodies in humans • 1938 - x-ray sections or slices (3D!) • 1912 - x-ray crystallography (Laue) - position of atoms in a crystal

37. History (5) - Computer Graphics • 1949 - SAGE air defense - tracked position of aircraft by radar, analyzed results and display on CRT • 1965 - sketchpad (Sutherland) - interactive graphical drawing system • Used to be BIG and EXPENSIVE

38. History (6) - Scientific Visualiz. 1987 - NSF report [McCormick87] • Personal/exploratory graphics - to enable a scientist to gain more knowledge (interact with data) • Peer graphics - enable scientist to show information to their colleagues and to collaborate • Presentation graphics - communicate information and results (high quality, fully annotated) • Publication of visualization - enable others to use the data (replicable)

39. History (7) - Augmented Reality. • 1983 - responsive environments (Myron Krueger)

40. Visualization Domains Volumetric data sources are usually produced by: • Scanning devices • Computation (mathematical), or • Simple measuring

41. Applications - Vis. As a Toolkit Application tools usually coupled with • Haptic feedback devices • Stereo output (glasses) • Interactivity demanding of the rendering algorithm

42. Scanning - Domains • Medical scanners (MRI, CT, SPECT, PET, ultrasound)

43. Scanning - Applications • Primary education • Medical education for surgery, anesthesia • Illustration of medical procedures to the patient

44. Scanning - Applications • Surgical simulation for treatment planning • Tele-medicine • Inter-operative visualization in brain surgery, biopsies, etc. • Industrial purposes (quality control, security) • Games with realistic 3D effects?

45. Scanning (2) • Domain - biological scanners, electronic microscopes, confocal microscopes • Apps - paleontology, microscopic analysis

46. Scientific Computation - Domain • Mathematical analysis • ODE/PDE (ordinary and partial differential equations) • Finite element analysis (FE), • Supercomputer simulations,

47. Scientific Computation - Apps • Computational fluid dynamics (CFD), • Computational field simulations (CFS),

48. Vector Field Viz Applications Computational Fluid Dynamics Weather modeling

49. Vector Field Visualization Challenges General Goal: Display the field’s directional information Domain Specific: Detect certain features Vortex cores, Swirl

50. Streamlines A curves that connect all the particle positions