Enhancing Usability: Pilot Study and Progress Presentation Exercise

1. Usability with ProjectLecture 14 – 31/10/08 Dr. Simeon Keates

2. Exercise – Part 1 • Last week you were asked to prepare your user trial protocols • Today – put them into practice • Perform a pilot study of the usability of your web-site with at least 1 user • Remember – the principal aim is to “test the test” • (or “trial the trial” or “evaluate the evaluation”…)

3. Exercise – Part 2 • Prepare a progress presentation for the board for Friday • Show that good progress is being made • Summarise: • The tasks performed • The data collected • Whether the user liked the site • Whether the user could use the site (e.g. complete the tasks) • What you think is working well in the design • What you think needs to be looked at more closely in the design • Any changes you would like to make to the site and protocol

4. Exercise - Practicalities • Remember to print out copies of your protocol • Allow plenty of blank space for adding observation notes • Allocate one person to do the pre-session briefing and debrief • Allocate one person to be the facilitator (the person who directs the user) • The remaining members act as observers

5. Cognitive models

6. The Power Law of Practice • Tn = T1 n-α • α = 0.4, T1 = 60s, T2 = 45.5s (24% faster), T10 = 23.9s (60%faster)

7. Cognitive modelling – Dealing with uncertainty • The Uncertainty Principle states that decision time T increases with uncertainty about the decision to be made: T = Ic H Where: H is the information-theoretic entropy of the decision; Ic = 150 [0~157] ms/bit • For n equally probable alternatives (Hick’s Law) : H = log2(n + 1) • More generally:

8. Cognitive modelling – The Model Human Processor Time_taken = x τp + y τc + z τm Where : x, y and z are integers τp = time for perceptual processor τc = time for cognitive processor τm= time for (simple) motor function

9. Motor skills – Fitts’ Law • A person wishes to hit this target: where: S x0 x1 x2 Start D

10. Merging the models One basic merged model is the Keystroke Level Model (KLM): Texecute = TK + TP + TH + TD + TM + TR • Where TK = total time spent keystroking = nk tk (# * time per stroke) • Time per stroke determined experimentally • TP = total time spent pointing (from Fitts’ Law) • Assume, say, 1.1 s per pointing action • TH = total time spent homing (moving hands between devices) • Assume 0.4 s per homing • TD = total time spent drawing = tD (nD, lD) (i.e. f(#, total length)) • Example: 0.9nD + 0.16lD • TM = total time to mentally prepare • Assume 1.35 s per preparation • TR = total system response time

11. Using the KLM • [Note: M = mental prep, K = keyboard, P = pointing] • Rule 0: Insert Ms in front of all Ks that are not part of argument strings proper. Place Ms in front of all Ps that select commands • Rule 1: If an operator following an M is fully anticipated in an operator just previous to M, then delete the M (e.g. PMK -> PK) • Rule 2: If a string of MKs belongs to a cognitive unit (e.g. name of a command), then delete all Ms but the first one • Rule 3: If a K is a redundant terminator (e.g. terminates a command immediately following the terminator of its argument), then delete the M in front of it • Rule 4: If a K terminates a constant string (e.g. a command name), then delete the M in front of it, but if the K terminates a variable string (e.g. an argument string) then keep the M in front of it

12. An more generic approach - GOMS The user’s cognitive structure consists of: • A set of Goals • A set of Operators • A set of Methods • A set of Selection rules

13. GOMS – a quick breakdown Goals: • Symbolic structures that define a state of affairs to be achieved • Examples: GOAL: EDIT-MANUSCRIPT or GOAL: MODIFY-TEXT • Goals can comprise sub-goals Operators: • Elementary perceptual, motor or cognitive acts whose execution is necessary to change any aspect of the user’s mental state or to affect the task environment • Examples: GET-NEXT-PAGE or GET-NEXT-TASK

14. GOMS – a quick breakdown Methods: • Procedures for accomplishing a goal – must be pre-learned at performance time (i.e. user already knows them) • Contain sets of Operators Selection rules: • Rules for helping the user decide which method to use to accomplish the goal • Example: if_such_and_such_is_true_then_use_method_M1_else_use_M2 To summarise: • Several Operators make up a Method, and • Selection rules are used to determine the best Method to reach the Goal

15. Using models of interaction • Fundamentally, you need to perform a comprehensive task analysis • The models indicate suggested performance for each sub-task • Those models help you to predict the performance of the interface • This can be used: • In design: Estimate performance using standard parameters to optimise your design • In usability trials: Estimate the performance and compare with actual observed data – investigate significant discrepancies

16. Extending to Universal Access applications – The MHP

17. Experiment I – Testing the MHP for motor-impaired users

18. Perceptual response times

19. Cognitive cycle times

20. Motor function times

21. Reaction times

22. Explaining the observed motor times (100-310 ms) • Theoretical interaction is: • Press the button (motor function) • Release button (motor function) • Consequently, either • very slow motor function times • or • extra steps being inserted

23. Identifying the delays • c & p calculated as for Experiment I • m button-down and button-up times separated • Motor function and reaction time tasks performed • Range of input devices used • mouse • touchpad button • space bar • EasyBall

24. The MS EasyBall

25. User descriptions

26. Results

27. Results

28. Results τm ?

29. PI7 results – Motor function task

30. Background ~c The MHP results

31. Conclusions • Extra cognitive cycles are being inserted • Interaction process is: • Decide to press button (cognitive) - OPTIONAL • Press the button (motor) - REQUIRED • Decide to release button (cognitive) - OPTIONAL • Release button (motor) - REQUIRED

32. Sources of extra cognitive steps • Users always in learning mode? • Users being overly careful? • Extra cognitive load from impairment?

33. Implications for use of user models • Individual components were comparable • However • method of combination was not • Therefore • need to verify user model assumptions before use

34. Implications for design • Users “add” own extra cognitive load • Need to support users by: • Minimising user uncertainty • Minimising cognitive load from program • Maximising interface intuitiveness • Maximising useful feedback

35. Extending to Universal Access applications – Cursor control

36. Symptoms associated with ageing and Parkinson’s Symptoms: • Essential tremor • Restricted motion • Reduced strength • Poor hand-eye co-ordination • Fatigue

37. Examples of motor-impaired cursor control

38. Cursor movement theories • Fitts’ Law • Relates target distance and width to time • Movement Optimization Model • Initial, pre-planned ballistic move • (Optional) Secondary corrective submovements • Submovements based on visual feedback • Analysis of movement paths • Describes effect of changes in distance, width and height of target • Longer distances => higher peak velocity • Smaller target => longer deceleration phase • Initial studies [Hwang et al., 2004] suggest NOT universally applicable

39. Cursor measures(MacKenzie et al - CHI 2001) • Target Re-Entry (TRE) • Task Axis Crossing (TAC) • Movement Direction Change (MDC) • Orthogonal Direction Change (ODC)

40. Cursor measures (cont.) • Movement Offset (MO) • mean deviation of points from task axis ( y ) • signed • Movement Error (ME) • average deviation of points from task axis • unsigned • Movement Variability (MV) • standard deviation of points from task axis • Missed Click (MCL) • Path Length / Task Axis Length (PL/TA) Additional measures

41. Cursor measures (cont.) • Can distinguish between motor impaired and able-bodied users • As “groups” • Keates et al. ASSETS 2002 • Can they do more? • Designed to explain why differences exist

42. User trials - The users • 4 groups of users • IBM interns (Y) – mean age 23, SD = 2.0 • IBM regulars (A) – mean age 47, SD= 9.4 • Older adults (OA) – mean age 79, SD = 4.5 • APDA members (P) – mean age 57, SD = 5.2 • 6 users per group

43. User trials - The experimental methodology • Fitts’ Law type task • 3 target sizes • 3 target distances • 36 target acquisitions per target session • 4 of each size/distance combination • Random angle of approach to target • 4 target sessions per user session (144 target acquisitions) • Interviews between each target session • Post-session debrief

44. User trials – Qualitative results • 21 difficulties reported with mouse use, e.g.: • Keeping hand steady when navigating • Slipping off menus • Losing the cursor • Moving in the desired direction • Running out of room on the mouse pad • Mouse ball getting stuck (and/or dirty) • 12 compensatory strategies, e.g.: • Avoid use of menus • Switch hands • Consciously go slower • Pause before clicking

45. Peak velocities No. of incorrect clicks User trials – Quantitative results Target activation times

46. User trials – Cursor measures (cont.)

47. No. of pauses >100 msec No. of pauses >250 msec User trials – Nature of movement observed • Differences in peak velocity do not explain all of target activation time differences • Theory: Target user movements are like able-bodied movements only more of them needed to complete the task

48. Normalised measures User trials – Nature of movement observed • Submovements can distinguish between user groups (p<0.01) • Submovements are significantly related to: • Path length / task axis length (PL/TA) • Missed/incorrect clicks (MCL) • Task axis crossings (TAC) • Target re-entries (TRE) • Movement direction changes (MDC) • Orthogonal direction changes (ODC) • Submovement not significantly related to: • Movement error (ME) • Movement offset (MO) • Movemenet variability (MV) Cumulative measures

49. User trials – Pauses and cursor measures

50. User Trials – Where the pauses occur