Interaction Devices

1. Interaction Devices CIS 577 Bruce R. Maxim UM-Dearborn

2. User Communication • Keyboard remains the user’s primary input device • Most GUI systems expect pointing devices as well • Light pen (direct) • Touch screen (direct) • Mouse • Trackball • Joystick • Graphics tablet • Touch pad • Foot controls (novel) • Gloves (novel) • Eye movement detectors (novel)

3. Pointing Device Tasks • Selection • Positioning objects • Orienting objects on screen • Path tracing • Quantify • Text entry/editing (not easily done)

4. Direct vs Indirect • Light pens and touch screens allow the fastest user responses, but are the least accurate (finger vs pixel level resolution) • Graphics tablets with a stylus allows good resolution and long periods of use without switching to the key board (with good software design – often menu driven)

5. Comparison • Keyboard • Fast control, require memorization and recall • Mouse • Best choice for pointing to objects in arbitrary positions, can be as fast as pointing with a finger • Joystick • Poor cursor control devices, actually slower than mouse (only use where they are game realistic)

6. Speech Technology • Discrete word recognition • Continuous speech recognition • Speech store and forward • Speech generation

7. Discrete Word Recognition • 90 to 98% reliability for small vocabulary • Usually requires speaker dependent training • Most people would rather type than dictate

8. When should you use it? • Speakers hands are busy • Mobility required • Speakers eyes are occupied • Harsh or cramped conditions prevent use of key board

9. Speech Store and Forward • Voice mail type technology • Video games • Low cost • Resource intensive

10. When to use computer generated speech? • Message is simple • Message is short • Message will not be referred to later • Message deals with events in time • Message requires immediate response • Visual communications channels are overloaded • Environment lighting is bad • User must move around • User subjected to high G forces or lack of oxygen

11. Display Technologies • CRT • raster scan cathode ray tube • LCD • flicker free, lightweight, limited viewing angles • Plasma • early touch screens • LED • some resolution limitations • Projectors • room lighting and resolution restrictions

12. Video Display Features • Rapid operation • Reasonable size (24 by 80 or 66 by 166 chars) • Reasonable resolution (768 by 1024 minimum) • Quiet operation • No waste paper • Low cost (< $100) • Reliable • Good for highlighting • Good for graphics and animation

13. Image Technologies • Scanners (< $100) • Digital cameras (< $200) • Digital video (< $200) • 12 inch video disk • CD ROM (600 MB data, 1 hour music) • DVD (3 GB data, 2 hours video) • Motion Capture

14. Hardcopy Devices • Printers • ink jet, laser, thermal, dot matrix • Plotters • seem to be vanishing (slow, high resolution) • Photographic printers • ink jet technology, special paper

15. Color • Can soothe or strike the eye • Adds accents to an uninteresting display • Facilitates subtle discriminations in complex displays • Emphasizes the logical organization of information • Draws attention to warnings • Can evoke strong emotional reactions of joy, excitement, fear, or anger

16. Color Display Problems • Colors can't be represented perfectly on digital computer systems • People don’t have the same perceptions of colors • Color blindness • Eye fatigue can be caused by using bad color combinations

17. Color Description Schemes • Computer Industry Approach (RBG) • Red, Green, Blue (0 – 255), additive model • Ink jet Approach (CMY or CMYK) • Cyan, Magenta, Yellow, subtractive model • Broadcast TV (YIQ) • Y = luminence, I, Q = chromacity • Psychological Dimensions (HSB or HSV) • Brightness (amount of light energy) • Hue (predominant wave length) • Saturation (color purity)

18. RGB Model

19. RGB Model

20. RGB Model • Note that for the "complementary" colors the sum of the values equals white light (1, 1, 1). • For example: red (1, 0, 0) + cyan (0, 1, 1) = white (1, 1, 1) green (0, 1, 0) + magenta (1, 0, 1) = white (1, 1, 1) blue (0, 0, 1) + yellow (1, 1, 0) = white (1, 1, 1)

21. CMY Model

22. CMY Model

23. CMY Model • Cyan = Green + Blue, so light reflected from a cyan pigment has no red component, i.e., the red is absorbed (or subtracted) by cyan. • Magenta subtracts Green • Yellow subtracts Blue. • Printers usually use four colors: Cyan, Yellow, Magenta and Black. • Cyan, Yellow, and Magenta together produce a dark gray rather than a true Black.

24. RGB to CMY C = 1 – R M = 1 – G Y = 1 – B CMY to RGB R = 1 – C G = 1 – M B = 1 – Y For printing (CKYK) K = min(C,M,Y) C = C – K M = M – K Y = Y – K Where K = Black CMY Conversions

25. RGB to YIQ |Y| |0.299 0.587 0.114| |R| |I| = |0.596 -0.275 –0.321| * |G| |Q | |0.212 –0.528 0.311| |B| • For YIQ to RGB use inverse matrix

26. HSV Model

27. HSV Model • Pure Red is H=0, S = 1, V = 1 • White is S = 0, V = 1 (H is irrelevant) • Black is V = 0 (both H and S are irrelevant) • Adding white is done by decreasing S and leaving V unchanged • Changing shades is done by decreasing V and keeping S unchanged • Create tones by decreasing both S and V

28. Limit total number of colors Recognize power of color to speed up or slow down a task Ensure color coding supports user task Have color coding appear with minimal user effort Place color coding under user control Design for good monochrome display first Use color to help formatting Using Color Effectively - 1

29. Using Color Effectively - 2 • Use consistent color coding • Be alert to user expectations for certain colors (e.g. red means stop or danger) • Use color changes to indicate state changes • Use color in graphic displays to increase information density • Be alert to bad color pairings (e.g. saturated blue and red) • Consider needs of color impaired users

30. Direct-Input • Create main DirectInput interface using CreateIDirectIntput8( ) • (optional) Query for device GUID’s for all devices (mouse, keyboard, joystick) • Create each device using CreateDevice( ) please note: GUID_SysKeyboard and GUID_SysMouse are built in • Set cooperation level for each device using SetCooperativeLevel( ) • Set data format for each device using SetDatFormat( )

31. Direct-Input • Set properties of each device using IDirectIntputDevice8::SetProperty( ) • Acquire each device using IDirectIntputDevice8::Acquire( ) • (optional) Poll devices using IDirectIntputDevice8::Poll( ) • Read device using IDirectIntputDevice8::GetStateDevice( )

32. Input • DirectX can request input in two ways • Immediate input (default) • Buffered input • LaMothe only uses immediate input • DirectX can cut Windows out of the message loop altogether, so be careful when you set each device cooperation level

33. LaMothe Example

34. Keyboard Global Declarations #include <ddraw.h> // directX includes #include <dinput.h> #include "T3DLIB1.H" // directinput globals LPDIRECTINPUT8 lpdi = NULL; // dinput object LPDIRECTINPUTDEVICE8 lpdikey = NULL; // dinput keyboard LPDIRECTINPUTDEVICE8 lpdimouse = NULL; // dinput mouse LPDIRECTINPUTDEVICE8 lpdijoy = NULL; // dinput joystick GUID joystickGUID; // guid for main joystick char joyname[80]; // name of joystick // these contain the target records for all di input packets UCHAR keyboard_state[256]; // contains keyboard state table DIMOUSESTATE mouse_state; // contains state of mouse DIJOYSTATE joy_state; // contains state of joystick

35. Game_Init( ) // initialize directdraw DDraw_Init(SCREEN_WIDTH, SCREEN_HEIGHT, SCREEN_BPP); // first create the direct input object if (DirectInput8Create(main_instance,DIRECTINPUT_VERSION, IID_IDirectInput8, (void **)&lpdi,NULL)!=DI_OK) return(0); // create a keyboard device ////////////////////////////////// if (lpdi->CreateDevice(GUID_SysKeyboard, &lpdikey, NULL)!=DI_OK) return(0); // set cooperation level if (lpdikey->SetCooperativeLevel(main_window_handle, DISCL_NONEXCLUSIVE | DISCL_BACKGROUND)!=DI_OK) return(0); // set data format if (lpdikey->SetDataFormat(&c_dfDIKeyboard)!=DI_OK) return(0); // acquire the keyboard if (lpdikey->Acquire()!=DI_OK) return(0);

36. Game_Shutdown( ) // kill the reactor Destroy_Bitmap(&reactor); // kill skelaton Destroy_BOB(&skelaton); // release keyboard lpdikey->Unacquire(); lpdikey->Release(); lpdi->Release(); // shutdonw directdraw DDraw_Shutdown();

37. Game_Main( ) // get player input // get the keyboard data lpdikey->GetDeviceState(256, (LPVOID)keyboard_state); // reset motion flag player_moving = 0; // test direction of motion if (keyboard_state[DIK_RIGHT] && keyboard_state[DIK_UP]) { // move skelaton skelaton.x += 2; skelaton.y -= 2; dx=2; dy =- 2; // set motion flag player_moving = 1; // check animation needs to change if (skelaton.curr_animation != SKELATON_NEAST) Set_Animation_BOB(&skelaton,SKELATON_NEAST); } // end if …

38. Mouse • Detects changes in X, y position (Mickeys) • Specifies “absolute” position • Can specify velcoity range • Good for selecting objects, moving objects, or changing head position (view) • Hard to center if used as joystick replacement • Has small number of buttons (2 or 3

39. LaMothe Example

40. Mouse Global Declarations #define BUTTON_SPRAY 0 // defines for each button #define BUTTON_PENCIL 1 #define BUTTON_ERASE 2 #define BUTTON_EXIT 3 // directinput globals LPDIRECTINPUT8 lpdi = NULL; // dinput object LPDIRECTINPUTDEVICE8 lpdikey = NULL; // dinput keyboard LPDIRECTINPUTDEVICE8 lpdimouse = NULL; // dinput mouse LPDIRECTINPUTDEVICE8 lpdijoy = NULL; // dinput joystick GUID joystickGUID; // guid for main joystick char joyname[80]; // name of joystick // these contain the target records for all di input packets UCHAR keyboard_state[256]; // contains keyboard state table DIMOUSESTATE mouse_state; // contains state of mouse DIJOYSTATE joy_state; // contains state of joystick

41. Game_Init( ) // first create the direct input object DirectInput8Create(main_instance,DIRECTINPUT_VERSION,IID_IDirectInput8, (void **)&lpdi,NULL); // create a mouse device ///////////////////////////////////// lpdi->CreateDevice(GUID_SysMouse, &lpdimouse, NULL); // set cooperation level lpdimouse->SetCooperativeLevel(main_window_handle, DISCL_NONEXCLUSIVE | DISCL_BACKGROUND); // set data format lpdimouse->SetDataFormat(&c_dfDIMouse); // acquire the mouse lpdimouse->Acquire();

42. Game_Init( ) // set the global mouse position mouse_x = screen_height/2; mouse_y = screen_height/2; // load the master bitmap in with all the graphics Load_Bitmap_File(&bitmap8bit, "PAINT.BMP"); Set_Palette(bitmap8bit.palette); // make sure all the surfaces are clean before starting DDraw_Fill_Surface(lpddsback, 0); DDraw_Fill_Surface(lpddsprimary, 0); // create the pointer bob Create_BOB(&pointer,mouse_x,mouse_y,32,34,1,BOB_ATTR_VISIBLE | BOB_ATTR_SINGLE_FRAME,DDSCAPS_SYSTEMMEMORY); // load the image for the pointer in Load_Frame_BOB(&pointer,&bitmap8bit,0,0,2,BITMAP_EXTRACT_MODE_CELL);

43. Game_Init( ) // set clipping rectangle to screen extents so mouse cursor // doens't mess up at edges RECT screen_rect = {0,0,screen_width,screen_height}; lpddclipper = DDraw_Attach_Clipper(lpddsback,1,&screen_rect); // hide the mouse ShowCursor(FALSE);

44. Game_Shutdown( ) // first unacquire the mouse lpdimouse->Unacquire(); // now release the mouse lpdimouse->Release(); // release directinput lpdi->Release();

45. Game_Main( ) // move the mouse cursor mouse_x+=(mouse_state.lX); mouse_y+=(mouse_state.lY); // test bounds // first x boundaries if (mouse_x >= screen_width) mouse_x = screen_width-1; else if (mouse_x < 0) mouse_x = 0; // now the y boundaries if (mouse_y >= screen_height) mouse_y= screen_height-1; else if (mouse_y < 0) mouse_y = 0;

46. Game_Main( ) // position the pointer bob to the mouse coords pointer.x = mouse_x - 16; pointer.y = mouse_y - 16; // test what the user is doing with the mouse if ((mouse_x > 3) && (mouse_x < 500-3) && (mouse_y > 3) && (mouse_y < SCREEN_HEIGHT-3)) { // mouse is within canvas region // if left button is down then draw if (mouse_state.rgbButtons[0]) { … };

47. Game Pads • Simplified keyboard • Buttons have good stimulus/response compatibility • Lets you use two hand • Good for multiplayer games • Can require weird combinations of buttons • Don’t get to rely on fastest finger (index) • Doesn’t support variable input

48. Joysticks • Lots of devices look like joysticks (digital – 8 positions, analog x,y deflection) • Allow variable input plus buttons • Great for specifying changes • Can be used to specify “rate in a direction” • Not good for specifying absolute positions (need to “hold position steady) • Not good for many arcade games

49. Using Joysticks & Game Pads • Scan for all devices and record GUID’s • Create joystick device using one GUID • Use interface from step 2 and create another interface (release the first) • Set cooperation level • Set data format and properties • Acquire joystick • Read joystick using Poll and GetDevice

50. LaMothe Example