The End of Pointing and Clicking: Improving Computer Access with Goal Crossing

1. The End of Pointing and Clicking: Improving Computer Access with Goal Crossing June 30, 2009

2. People

3. Design Criteria Ease of motor-impaired performance How easy is it motor-wise to acquire the target? Efficiency? No t No multiple steps Speed for able-bodied? Speed for motor-impaired? Safety (Unwanted targets should not be accidentally triggered) How easy/difficult is it to make a false selection? Tolerance to movement error? Is the design reversible?Is it cancelable? Can the user abort? Scalability Does it handle high-density layouts? How does it scale to hundreds of menu items, links, buttons, and icons? Is it compatible with conventional point-and-click widgets? Can both coexist? Etc. Are the benefits of crossing preserved (e.g., “follow-through,” “aggregation”)? Is it easy to learn? Does it require preexisting knowledge? How hard will it be to implement? Is heavy animation required for the technique to work? Can it lend itself to a nice visual design?

4. 1. Basic Crossing Ideas 1.1 Cross While Off-hand Button Held Down 1.2 Cross and Click Anywhere 1.3 Drag and Cross 1.4 Cross and Cross-back 1.5 Delta t

5. 1.1 Cross While Off-hand Button Held Down OK

6. 1.2 Cross and Click Anywhere Click! Click! OK

7. 1.3 Drag and Cross Drag & Cross OK

8. 1.4 Cross and Cross-back Cross goal to activate OK Activated Selected Double clicked (or) Drag (or) Right clicked … Cross back to confirm selection OK Cross goal to activate Cross goal to activate OK OK Cross back to confirm selection The target gives visual feedback OK OK

9. 1.4 Cross and Cross-back (cont.) Activate an icon by crossing over icon and initial bar OK Then back over, turn 90-degrees and then cross the two bars OK Byron McMullen . Jacob Nelson . Tara Tarjoto . Tim Wibowo

10. 1.5 Delta t OK Cross and stop for N seconds to confirm the selection

11. 1.5 Delta t (cont.) Cross and Stop

12. 2. Advanced Crossing Ideas 2.1 Secondary Goal Crossing 2.2 Gesture 2.3 Corners and Edges 2.4 Moving Targets 2.5 Features of Crossing Event

13. 2.1 Secondary Goal Crossing • BUTTON g2 first goal: button OK second goal (g2): crossing the g1 initiates g2 to appear OK OK OK Length-to-the-g2 variation angle variation OK OK OK OK OK G2 design variation: how to make it look like a goal?

14. 2.1 Secondary Goal uArc-shape Crossing the first target activates the arc OK (a) Location of the 2nd goal changes depending on the approaching angle

15. 2.1 Secondary Goal v L-Shape g2 fades out if they are not being selected during a certain amount of time Right-handed, left-handed

16. 2.1 Secondary Goal w I-Shape • MULTI-LAYERED MENU: Red bar appears only when the menu has no underlying menu items

17. 2.1 Secondary Goal x ‘dontclickit’ idea ANIMATION OK • GO OK • GO

18. 2.1 Secondary Goal y Cluster Interactive prototype Targets are clustered (3*3 or 2*2) and the secondary goals appear at the fixed location T1 T2 T3 T4 T6 T7 T8 T9 2 3 4

19. 2.2 Gesture uZig-zag / Scribble Nope, it’s not selected OK OK OK Yes Yes

20. 2.2 Gesture v Encircling Targets ANIMATION

21. 2.2 Gesture w Pigtail ANIMATION

22. 2.2 Gesture x Hover Widget cross Turning 90° activates the most recently crossed goal. When the mouse crosses the end of the tunnel, the crossed goal is confirmed cancel

23. 2.2 Gesture yShivering Around a Target Sparse layout, one small target

24. 2.2 Gesture  Rectangle Lasso Public Downloads File Folder To open Philip Phung, Colin Booth, Mark Vitazko, Chris Claiborne

25. 2.2 Gesture  Color In Public Downloads File Folder Philip Phung, Colin Booth, Mark Vitazko, Chris Claiborne

26. 2.2 Gesture  Letter gesture Chris Sinco, RishiTalwar, Justin Wilbourne, Ken Wong

27. 2.2 Gesture  OK OK Alexei Bespalov, Kevin Merritt, James Rodgers

28. 2.3 Corners and Edges u Constraining mouse movement with edges

29. 2.3 Corners and Edges v Now only Listen to X-Movements OK Only Listen to Y-Movements • Basic Idea: Person can make the computer ignore x or y movements, creating an edge to travel along. • The edges (or guidelines, or whatever we call them) could be activated with a keyboard/mouse button? Any other options? • Maybe both can be activated to keep the from moving during a click? • Could be used for crossing too.

30. 2.3 Corners and Edges w Box Click! Skip Select All Select Corners are easy to hit. Make a context menu out of a square, where corners are options.

31. 2.3 Corners and Edges x Cancel Open C.M OK OK

32. 2.3 Corners and Edges y Diamond Crossing one object Crossing more than one object Open Open Cancel Cancel Context Menu (C.M.) Previous Open Open Open all Cancel (Cancel) Cancel Context Menu (C.M.) Previous

33. 2.3 Corners and Edges z Hover Widget + Diamond Crossing one object Crossing more than one object Open current X Open multiple Context Menu X Open current Open Open multiple Context Menu C.M Open C.M • Hitting the corner triggers the queued up targets (aggregation) • Select or delete the targets that are in the queue

34. 2.3 Corners and Edges  90-degree OK OK

35. 2.3 Corners and Edges  TARGET

36. 2.3 Corners and Edges  The mouse can travel through either of the entrances/exits to begin activating the button, but the cursor must travel all the way through to the other side in order to fully trigger the button’s action. OK OK OK Once the cursor enters the button, barriers at the top and bottom of the button prevent the passage out of the button. These barriers exist only once the mouse moves in a vertical direction. Chris Sinco, RishiTalwar, Justin Wilbourne, Ken Wong

37. 2.3 Corners and Edges Bounding Buttons Certain elements have un-mousable “screen edges” affixed to them to avoid pointer overshoot. Ballistic movement can now hit and stick to targets. Nathan – Peter - Kaitlyn

38. 2.4 Moving target u Coming Toward Closest target is triggered, and then the target is coming toward the cursor

39. 2.4 Moving target v Stepping Backward Closest target is triggered, and then the target is stepping backward from the cursor

40. 2.4 Moving target w Rearranging Targets

41. 2.5 Features of Crossing Event u Velocity Not selected Selected Too slow Needs to cross above certain speed in order to select the target

42. 2.5 Features of Crossing Event v Acceleration Not selected Selected a= 0 Stop and then accelerate the crossing speed (A>0)

43. 2.5 Features of Crossing Event Angle Selected Not selected Orthogonal crossing: selected

44. 3. Other Crossing Ideas 3.1 Kinematic Template 3.2 Accessible Bubble Cursor 3.3 Switch 3.4 The Clock 3.5 Guideline 3.6 Trapezoid 3.7 Extended Crossing Line 3.8 Plough / Force Field Cursor 3.9 Edge.Point 3.10 Area Crosser 3.11 Stabbing 3.12 The Casual Fan 3.13 Triple Cross 3.14 Cross and Cross Back with Hover Widget 3.15 Corner Pocket

45. 3.1 Kinematic Template Passive template: the cursor goes through the button with the help of the (passive) template OK OK

46. 3.2 Accessible Bubble Cursor Tab through targets! OK OK OK OK Cancel Cancel Cancel Cancel Menu Menu Menu Menu Options Options Options Edit Edit Edit Edit Open Open Open

47. 3.3 Switch Gravity effect + visual feedback activated OK Switching on OK OK

48. 3.4 The Clock In this concept, a lever appears inside a circular overlay on the target when the cursor gets close to a goal. To trigger the selection of a target, the user “pushes” the lever (like a clock hand) in a circle until a full rotation is made. The cursor “sticks” to lever, facilitating rotation like the hand on a clock. Once the rotation movement has begun, the user can escape the “stickiness” by sliding the curser off the outer edge of the lever towards the outside of the circle. Clarke Freeman, Alex Jansen, Kristofer Martin, Josh Rakita

49. 3.5 Guideline Selected If the cursor follows the guideline for a certain amount of time and then cross, the target is selected Stop for a while -> guideline appears Not Selected Selected

50. 3.6 Trapezoid OK OK OK