Interaction Devices

1. Interaction Devices • Interaction Devices • The Standard: QWERTY (or Sholes) Keyboard • Mobile devices are driving the need for future input devices • Pointing devices (mouse, touchscreen) • Gestural input • Two-handed input • 3-d pointing • Voice input/output • Wearable devices • Whole body involvement • Niche applications • Eye-trackers • DataGloves • Haptic/force-feedback • Brain-controlled mouse movement

2. Interaction Devices • Interaction Devices • Multimodal interfaces • Combine several modes of input/output • E.g., voice commands with pointing devices • Likely direction • Giving users the ability to switch between modes depending on their needs • E.g., Driving a car • Operate navigation systems with touch or voice input • Invoke visual or voice output based on location (e.g., moving in traffic vs. at a stop sign) • DB adjustment for ambient noise • Context-aware computing • Sensors: • Global Positioning System (GPS)

3. Interaction Devices

4. Interaction Devices • Interaction Devices • Context-aware computing • Sensors: • Global Positioning System (GPS) • Cell-phone sources • Wireless connections • Make detailed information available about the users surroundings • Restaurants, gas stations • Museum visitors or tourists • Auto-connect to a printer based on room location • Yelp • Privacy issues

5. Interaction Devices • Keyboards and Keypads • QWERTY • Beginners approximate 1 keystroke per second • Average office worker is 5 keystrokes per second (50 words per minute)

6. Interaction Devices • Keyboards and Keypads • Potential for higher rates of information input • Courtroom recorders (300 words per minute) • Piano Keyboard • Allows several finger presses at once • Responsive to different pressures and durations • Chord keyboards • One-handed keyboards • Useful for tasks requiring one hand manipulation of an object

7. Interaction Devices • Keyboards and Keypads • Reduction of ulnar abduction and pronation • Ulnar tunnel syndrome • Ulnar tunnel syndrome is caused by pressure on the ulnar nerve at the wrist • This nerve is found on the pinkie-finger side of the wrist

8. Interaction Devices • Keyboards Layouts • QWERTY • Keep frequently used letters apart • Slow down users to avoid key jamming • Dvorak • Reduces finger travel time • Increases speed, reduces errors

9. Interaction Devices • Keyboards Layouts • Number Pads – phone pads versus calculator pads • Most computer keyboards use the calculator layout • Performance is slightly better with the phone layout

10. Interaction Devices • Keyboards Layouts • Keyboard for those with disabilities • Combination of small hand movements and small finger presses selects the letters and controls the cursor • No finger or wrist movement is needed • Helpful to users with carpal tunnel syndrome or arthritis • Each dome slides into one of eight zones to type a character • Either dome can slide first or move both at the same time. • Domes slide toward the center of their color or character zones. (not directly at the characters) • Slide the right dome to the zone of the character you wish to type; slide the left dome to the color of that character.

11. Interaction Devices • Keyboards Layouts • Dasher • Predicts probable characters and words as users make their selections • Continuous two dimensional pointing with a mouse, touchpad or eye-tracker • http://www.youtube.com/watch?v=K6jnY2bNIzw

12. Interaction Devices • Keys • Modern keyboards • ½ inch square • Slightly concave surfaces • Matte finish • 40 to 125 gram force • 3 to 5 millimeters of displacement (when the key is pressed far enough to send a signal, both tactile and audible • Larger keys (ENTER, SHIFT, CTRL) • Keys with indicators (CAPS LOCK, NUM LOCK) • Combination Keys (Ctrl-V) • Cursor movement keys • Other special keys (HOME, PAGE UP, END, PAGE DOWN) • Number Pad

13. Interaction Devices • Keyboards and Keypads for Small Devices • Foldable Keyboards • Virtual Keyboards

14. Interaction Devices • Keyboards and Keypads for Small Devices • Mobile Phones and Devices • Multi-tap to select a letter • Word prediction • Palm Graffiti

15. Interaction Devices • Pointing Devices • Avoids learning commands • Reduces the chances of typographical errors • Keeps the user’s attention on the display • Operations • Select • Position (e.g., move) • Orient (e.g., rotate) • Path (e.g., curving line in a drawing program) • Quantity (e.g., numeric selection) • Text (e.g., enter, move, edit) • Direct Control Devices • Examples: Light Pen, Touch Screen • Indirect Control Devices • Examples: Mouse, Trackball, Joystick

16. Interaction Devices • Pointing Devices • Criteria for success • Speed and accuracy • Learning time • Cost and reliability • Size and weight • Example: Touch screen • Gestures • Touch screen is the only input that has survived the Disney theme parks • Indirect Control Pointing Devices • Eliminate hand fatigue and hand obscuring the screen of direct input • Required more cognitive processing and eye/hand coordination • Examples: Mouse, trackball, joystick, graphics tablet

17. Interaction Devices • Comparison of Pointing Devices • Light pen and touch screen (fast but inaccurate) • Mouse (fast and accurate) • Keyboard entry in sometimes faster than the mouse (e.g., data entry) • Joysticks and trackballs are often preferred over mice by users with motor disabilities • Alternative keyboard entry should be provided • Touch screen and trackball are durable for public access • Mouse, trackball, graphic tablet and touch pad are good for pixel level pointing

18. Interaction Devices • Fitts’s Law • Predicts the amount of time to point at an object • Helps to decide the location and size of buttons • Helps to decide on the types of pointing devices as a function of task • The time for hand movements was dependent on the distance users had to move (D) and the target size (W) • Doubling the distance took longer, but not twice as long • Increasing the target size enabled the user to point at the object more rapidly MT = a + b log2(D/W + 1) a = approximates the start/stop time in seconds for a given device b = inherent speed of the device if a = 300 msec, b = 200 msec/bit, D = 14 cm, W = 2 cm then MT (movement time) = 900 msec

19. Interaction Devices • Fitts’s Law • The farther away a target is, the longer it takes to acquire it with the mouse • The smaller a target is, the longer it takes to acquire it with the mouse • The inverse of both statements is true as well (closer and bigger targets can be more quickly acquired) • There are two main ways to improve mouse efficiency: put the controls closer, or make them bigger • As monitors have gotten bigger and screen resolutions have increased, Fitts' law dictates that actual mouse efficiency has gone down • In other words, the same button takes much longer to click than it did fifteen years ago • Office 2007 and Fitts' Law • Most controls in the Ribbon are labeled. This helps discoverability and usability considerably, but it also makes the buttons bigger and easier to target.

20. Interaction Devices • Fitts’s Law • The Mini Toolbar was designed with Fitts' Law in mind as well. Whenever you select text or right-click selected text, a small toolbar appears directly next to the mouse cursor. As you move closer to it, it fades in; as you move away, it fades out. • The controls on the Mini Toolbar are small, but because they're located directly next to your cursor, they're easy to target

21. Interaction Devices • Fitts’s Law • Mile-High Menus and Magic Corners • One of the most useful aspects of applying Fitts' Law to computers is that screen size is bounded. No matter how far you move your mouse to the left, the cursor will never go farther than the left side of the screen • Because the Quick Access Toolbar is in the title bar, the buttons are effectively infinitely tall. You can target and click each of the buttons very quickly; they're a "mile high." Office Button

22. Interaction Devices • Novel Devices • Foot Controls – Foot Mouse (takes twice as long as a hand operated mouse) • Eye-tracking – still mostly a research tool • Data Glove • Haptic Feedback – moving a control (e.g., a mouse) and feeling resistance • Bimanual Input – non-dominant hand makes a coarse selection, the dominant hand performs a precise movement • Ubiquitous Computing • Embeds sensing technologies in a room • The user wears badges so the sensors detect the presence of a user • Application: tracking patients in a hospital

23. Interaction Devices • Novel Devices • Digital Pen (by Logitech) • Records the strokes written on digital paper • When the pen is placed back in its holder, the data is transferred to the computer • Digital Microscopes • Stores magnified pictures and annotations

24. Interaction Devices • Speech and Auditory Interfaces • The vision of computers chatting with users may be more of an uninformed fantasy than a desirable reality • Voice commanding is more demanding of user’s working memory than is hand/eye coordination • Speech requires use of limited resources, while hand/eye coordination is processed elsewhere in the brain, enabling a higher level of parallel processing • Planning and problem solving can proceed in parallel with hand/eye coordination, but they are more difficult to accomplish while speaking • Variations related to speech and audio technologies • Discrete-word recognition • Continuous-speech recognition • Voice information systems • Speech generation • Non-speech auditory interfaces

25. Interaction Devices Siri • Speech Systems • Potential uses • Users with vision impairments • When the user’s hands are busy • When mobility required • When the speaker’s eyes are occupied • When harsh or cramped conditions preclude use of a keyboard • Technologies • Speech store and forward • Discrete-word recognition • Voice information systems • Speech generation • Issues with speech recognition • Increases cognitive load when compared to pointing • Speaking commands or listening disrupts planning and problem solving • Interference from noisy environments http://www.youtube.com/watch?v=EySAW1C60is&feature=related

26. Interaction Devices • Speech Systems • Issues with speech output • Slow pace of speech output when compared to visual displays • Ephemeral nature of speech • Difficulty in scanning/searching

27. Interaction Devices • Discrete-word recognition • Recognition of individual words • Accurate 90% to 98% for 100 to 10,000 word vocabularies • Speaker-dependent training (the user trains the system) • Speak-independent (not as accurate) • Keyboards and pointing devices are more rapid, and actions and command are more visible for easy editing • Error handling is difficult and slow • Continuous-speech recognition • Difficult to recognize the boundaries between spoken words • Problems with accents, variable speaking rates, disruptive background noise, and changing emotional conditions • Fields with distinct terminology may be able to use this technology due to the limited and precise vocabulary

28. Interaction Devices • Voice information systems • Interactive Voice Response (IVR) • Examples: Help support, banking, ordering, voice mail • Challenges • Complex, deep structures • Slow pace of voice output • Difficulty scanning • Apple’s iPod • Management of large audio databases • Retrieval of specific segments

29. Interaction Devices • Speech Generation • Automobiles systems (e.g., “Turn left onto Route 896) • Control rooms (e.g., “Danger, temperature rising”) • Privacy related to audio messages • Is the human voice a better means of communication (e.g., Atlanta Airport)? • Are tones sometimes better than voice (e.g., “Headlights on”) • Microsoft’s Windows Narrator (reads text display on screens) • Voice tagging of web pages (VoiceXML) • Googles TalkBack • Non-Speech Auditory Interfaces • MIDI (Musical-instrument digital interfaces) for music composition • Auditory icons – distinctive familiar sounds • Earcons – abstracts sounds whose meanings must be learned • Example Earcons http://upload.wikimedia.org/wikipedia/commons/0/07/Emergency_broadcast_system.ogg http://www.youtube.com/watch?v=IvEwOfL21Uo

30. Interaction Devices • Displays – Small and Large • The primary source of feedback to the user • Physical dimensions (diagonal) • Resolution (number of pixels) • Number of available colors • Luminance, contrast and glare • Power consumption • Refresh rates • Cost • Reliability

31. Interaction Devices • Displays Technologies • Raster-scan cathode-ray tubes (CRTs) • Similar to televisions • Electron beams sweeping out lines of dots to form letters and graphics • Liquid-crystal displays (LCDs) • Voltage changes influence the polarization of tiny capsules of liquid crystals

32. Interaction Devices • Display Technologies • Plasma display panels (PDPs) • Rows of horizontal wires are slightly separated from vertical wires by small glass-enclosed capsules of neon-based gases • When the horizontal and vertical wires receive a high voltage, the gas glows • Light-emitting diodes (LEDs) • Certain diodes emit light when a voltage is applied • The curved display in New York’s Time Square

33. Interaction Devices • Displays Technologies • Large Displays • Informational wall displays (control room usage)

34. Interaction Devices • Display Technologies • Large Displays • Interactive Wall Displays • SMART board

35. Interaction Devices • Display Technologies • 3D Smart Phones • Visual Strain?

36. Interaction Devices • Display Technologies • Perceptive pixel on CNN • Check this out: CNN Perceptive Pixel • Notice • Gestures • Resizing capabilities • Selection capabilities • A world of Glass • http://www.youtube.com/watch?v=bBjvqnKQsTI&feature=related

37. Interaction Devices • Display Technologies • Multiple desktop displays • Six 19” Monitors • Facilitate side-by-side comparisons of documents, software debugging, information visualization and analysis

38. Interaction Devices • Display Technologies • Heads-up and helmet-mounted displays • Project information on a partially silvered windscreen of an airplane cockpit or car • Operators can remain focused on the surroundings while receiving computer generated information • Mobile device displays • Making phone calls • Texting • Checking appointments • Reading email • Finding a locations • Obtaining directions • Listening to music • Internet browsing https://www.youtube.com/watch?v=1oyCzT6sB_4 5 minutes into video

39. Interaction Devices • Display Technologies • Printers • Speed • Quality • Cost • Compactness • Quietness • Type and size of paper • Support for special formats • Reliabilty http://www.youtube.com/watch?v=KGKJQ2WMTdY

40. Interaction Devices • Companies are starting to sell products that make things • Don't need a degree in electronics or programming to create something. • 3D printers • Deposit tiny beads of melted plastic, solidify resin with ultraviolet light. • Material: because the robot arm deposits a mix of thermoset polymers that harden at the exact same time they're pushed out the tip of the extruder • http://www.theverge.com/2013/5/28/4371156/mataerial-3d-printing-anti-gravity-object-modeling-aom