DGL / 3DITs presentation

1. DGL / 3DITs presentation By Andrew A. Ray

2. Schedule • 9:00-10:00 - DGL – Overview and programming guide • 10:00-10:10 - Break • 10:10-10:30 - Discussion on potential usages of DGL with your software projects • 10:30-11:00 - Overview of available VR hardware • 11:00-11:30 - Introduction to 3D Interaction techniques • 11:30-11:40 - Break • 11:40-12:00 - Discussion and final wrap-up • 12:00 - Lunch

3. DGL • Capabilities • Case studies • Output • OpenGL, Open SceneGraph, Coin, VTK • Input • Mouse, keyboard • Communication with other processes

4. Capabilities of DGL • Extends DIVERSE’s reach into OpenGL based rendering and toolkits built to render to OpenGL • Primary thrust is to provide VR capabilities to Open Scene Graph, while allowing more flexibility than DPF did with OpenGL Performer • Improves upon some parts of the DPF API

5. Case study 1 • Road crossing experiment using VRML • Elderly citizens crossing the street in a city • Controlled environment with no unreasonable risk for the participant • VRML animation for cars / streetlights • Coin used to render the project • Coinfly example was all it took to run the project

6. Case study 2 • Benefits of immersion study • 3D mathematical equations • Different levels of immersion (walls on / off) • Sometimes head tracker used / sometimes not • Used to help understand 3D math and what it looks like in its natural surroundings

7. Case study 3 • AMADEUS – Underground mining visualization tool • Designed to view statistically generated fractures to simulate what the rock looks like before it is tunneled through • Somewhat simple novel computer graphics algorithm to cut different types of the tunnels out of the rock and be able to see them in true 3D

8. Screenshot

9. Result of graphics algorithm

10. DGL Output capabilities • Based around a draw callback that is added to a dtkAugment • Allows for OpenGL to be drawn before and after navigation and has an overlay capability to ensure that it is drawn after all the other OpenGL • How does this model work with Open SceneGraph / Coin / VTK?

11. DGL Augment Interface • Same exact process as making your own dtkAugment • Derive off the draw callback • Pass in the type of dglAugment you want it to be (i.e. where it is drawn) • Load the DSO into your favorite application • Done

12. Porting existing programs to DGL • DGL is designed to ease porting old OpenGL programs. • There are C based wrappers that offer the same functionality as the C++ interface • dglDisplayCallback • dglInitGLCallback • dglPostframeCallback • dglSet/GetData

13. Problems with porting old GL programs • Timers aren’t implemented in DGL • Input probably is going to have to be redone • Depending on the application, it may have to be re-architected a bit to handle cluster development • May also have to worry about multi-context issues

14. Open Scene Graph usage • Step 1 – Develop OSG application • Step 2 – Initialize DOSG (DOSG::init()) • Step 3 – Insert top level OSG root node into DOSG::getWorld()->addChild(here); • Done

15. Open Scene Graph • DGL handles structuring Open Scene Graphs internals so that all you have to do is add a child to a node in a scenegraph that is drawn by DGL • Structure of scenegraph is scene nav ether world

16. Coin • Very similar to osg, except there is no nav node scene world ether • DCoin::init – Make the system • DCoin::getWorld()->addChild(here) • DCoin::loadFile(“model.iv/wrl”);

17. VTK • 4 step process • Create a render window (DVtkRenderWindow::New()) • Create a renderer (DVtkRenderer::New()) • Add the renderer to the window (AddRenderer(pointer)) • Add your actors to the scene • Flip the scene to OpenGL coordinates if desired (DVtkRenderer->RotateSceneX(90))

18. Input for DGL • Now that we have graphical development taken care of, we need to get how the user develops applications! • Same as DPF for wand / joystick / buttons / head tracker (dtkSharedMemory) • Has a different mouse / keyboard interface due to a different input system (Producer)

19. Mouse input • DGLMouse and DGLMouseButton • DGLMouse allows for setting the mouse position, and reading the mouse’s current position or the queued position of the mouse (setMouse(x,y); readMouse(float* x, float* y); getMouse(float* x, float* y);

20. Mouse Input • DGLMouseButton – Tells you what happened to the mouse button, and the position of the mouse when the button action happened • Based on top of shared memory • Concept of Actions, state for each mouse button • int readMouseButton(x,y, action) • Returns 0 if it doesn’t have data, otherwise it tells you the position it was changed at and what the change is • Actions are press / release / unknown • Allows for the ability to fake mouse events if there is a need

21. Keyboard Input • It uses the same actions concept as mouse input did • DGLKeyboard::getState(DGLKeyboard::getKeyCharacter(“a”)); • Pass an action pointer (to be written to) into readKeyCharacter and it will return the action / character that was pressed • Do have to switch between char and KeyCharacter due to Producer syntax

22. The raw deal • If you don’t like the helper classes created then there is always shared memory. • It is what the system is based off of. • dgl/mouse dgl/keyboard dgl/mousebutton are the segments • The structure is usually an integer for the data and an integer for the action

23. Nuts and bolts • Building dgl programs is the same as with dpf programs, dgl-config is your friend • dgl-config --include and --libs • dgl-config --osg-include --osg-libs • dgl-config --coin-include --coin-libs • dgl-config --vtk-include --vtk-libs • Example makefiles in the examples directory on how they are used

24. Design strategies for DGL programs • Decide the type program that you are developing – OSG , OpenGL, etc… • Decide the level of interactivity • Often having a DSO that reads / displays data is the best approach because you can reuse DGL shell programs (i.e. dosg-fly) • Full applications give the most flexibility and opportunity

25. DGL design strategies • Often I break the problem into these specific parts • Reading input data • Transforming input data • Displaying data • Interacting with data • Often need to be careful about structure of data storage / accessibility because these decisions often interactivity

26. DGL design strategies • Often interactivity places the most demands on applications • The earlier in the development process that you consider it, usually the best • Aim for several quick iterations that can test different features / abilities • Get users involved as soon as possible in the development cycle

27. Questions / Comments • Capabilities? • Design decisions? • Application design? • Building applications?

28. 10 minute Break

29. VR Hardware • Often what you do in VE creation is heavily influenced by the input devices you have available • Very large range of devices, some very well supported, some not so much. • In CAVE environments usually light weight one handed devices are preferable • Often have to consider how equipment will affect others in the CAVE environment (for demos)

30. Outline for VR hardware • Text entry • Positional trackers • Pointing devices (other than wands) • Niche areas • Haptics

31. One handed text entry A one handed text entry method through chording Press multiple buttons to obtain different letters of the alphabet Advantages: Only needs 16 keys to operate Can be used as a cheap input device that has 16 different buttons Can be combined with a velcro’d tracker Disadvantages: Requires training Will not be as fast as regular keyboard text entry

32. Frog pad

33. Software text entry On screen keyboards are often used for text entry. Basically make a rectangle with all of the different keys you want. Stick a ray on the end of the wand, put the ray into the letter you want. Press a button on the wand to select the letter.

34. TULIP • Tulip menus / text entry • Imagine wearing pinch gloves and having a floating keyboard in front of you in the world • Reach out and pinch the letter in front of you • 11 -12 WPM for expert users

35. Optical tracking Different way of tracking Put markers (little balls) on a set of glasses or a hat. The camera then recognizes them and generates a 3D position based on the Info. Advantages: Cheaper than Intersense Can easily make any object trackable Disadvantages: Requires users to wear something Marker has to be visible to camera

36. Magnetic tracking • Tracking through electromagnetic fields • Doesn’t work well with metal objects in the environment • Higher latency than with Intersense trackers • More portable though

37. Acoustical tracking Acoustical tracking Low latency Not sensitive to metal in the environment Not cheap or portable (has to be calibrated for each environment. Very good for fixed setups such as CAVEs

38. Computer vision tracking • Through the use of markers and a standard webcam you can get 6DOF tracking • Software process • Useful and cheap for passive haptics

39. Gyroscopic mice The gyroscopic mice provide 2D in air movement and generally do not require drivers, just plug in and the OS / USB does the rest (even on Linux) Advantages: -Allows for a cheaper ($70) more restricted wand movement (only desktop X,Y) -Is possible to have multiple mice influencing the VE at the same time -Scenario (All data on front wall, multiple clickable options). Disadvantages: -Does not have absolute tracking -Has to be used with care, a flick of the wrist during a demo could be problematic

40. Pinch Gloves Pinch gloves allow for fingers to be pinched together to produce different combinations for input into the VE. Combine with a 6DOF tracker to know where the users hands are. Somewhat cheap: $2,000 device Advantage: More natural style of interaction than with a wand Disadvantage: Does not track finger movement

41. Data gloves • Allows for true finger positioning to be reported to the VE • Can do sign language type gestures • Has calibration issues • More expensive • Doesn’t always work right • Very expensive

42. Shape tape • A piece of material that can be shaped and have this shape reported to the computer • Useful for art, engineering (measuring / curve placement) • Can be used for interfaces when combined with trackers • Also can be used to imitate snakes

43. Omni Directional Treadmills Imagine being able to walk around in an environment naturally and not have to worry about the running into the CAVE walls. Advantages: Natural movement in a VE Perfect for types of training Disadvantages: Expensive Noisy May be harder than regular walking and may not react well to running

44. Haptic devices • Provide force feedback • Sometimes in a pen form • Sometimes a skeleton framework over a glove • Often easier to use passive haptics (pit experiment)

45. Discussion on Input devices • Would adding a different input device be useful for your application? • Make for interesting demos • Can even go so far as hang-gliders with fans / tilt recognition or walking in place harnesses / floor tiles that move around • Questions / Comments

46. 3DIT • Interacting with VEs makes them not only more usable, but more useful • Helps increase the sense of presence • Many different types that provide very interesting abilities.

47. 3DIT Input • Require input devices to work • Does not require a 3D input device to do a 3D task (mouse / virtual sphere) • Input devices do enable / limit different types of input though

48. 3DIT Realism • Natural versus Magic techniques • Think of having a virtual cup of water on the table in front of you. • Reach out and grab it • Now imagine it being 15 feet in front of you • Both are possible in 3DITs • Can be mixed to have some of both

49. 3D Selection and Manipulation • Tasks are mainly composed of selection, positioning, and rotation • Selection is composed of indicating which object is desired, confirming that it is the object that is desired to be selected, and then providing feedback to the user. • Can be done through a couple of different methods (ego / exo centric)

50. Different types of selection techniques • Simple infinite (or finite) ray to select an item • Two handed pointing • Flashlight / aperture addition to raycasting • Flexible ray casting • Image plane techniques • Fishing reel technique