NGOMSL Modeling

1. NGOMSL Modeling Psyc645 -- Week 10 Wayne D. Gray and Thomas Mayfield

2. Next Week -- TUESDAY!! • NGOMSL#1 • Produce NGOMSL methods from the trace of the VCR (my trace not yours) • Are no pops in NGOMSL. Verifies may be suitable replacements. Yours to decide. • NGOMSL#2 • Create NGOMSL methods for the ManTel task • Should be one set of methods that can be applied to both interfaces and to 1 or 2 queries • Obviously, not all methods need to be used in each analysis • Produce an NGOMSL trace for Dialog-Box 1 query and PopUp 2 queries • Readings • Paul Green’s 1999 Human Factors Ergonomics Society conference article

3. Psyc645 -- Reprise of GOMS lecture 1 • Levels of analysis based on Newell’s Time Scale of Human Action • Important to concept is that behavior at one level can be decomposed into behavior at next lower level • Show what is added at various levels • Important to the argument that I am trying to make to convince you that analyses at increasing lower levels entail decreasing degrees of generality (or increasing degrees of specificity) • Programming VCR is a generally understood concept, but programming my VCR is very different than programming George Lukas’ VCR. Setting Tom’s watch or Mary Ann’s watch is very different than setting the clock in my car • Implications? • Behavior is not as unconstrained as we would like to believe. As we get more and more specific in our description of behavior we find more and more constraints on behavior.

4. Psyc645 - Overview of GOMS lecture 2 • NGOMSL analyses • Unlike KLM, NGOMSL is a serious notation for discussing interactive behavior at levels lower than the unit task • Not simply a tool for analyzing design but a vehicle for representing low-level performance • Today we will work through several examples

5. NGOMSL • Natural Language GOMS • based on structured natural language notation and a procedure for constructing them • models are in program form • Control Structure: Hierarchical goal stack • Serial or parallel: Serial • Level of Analysis: • In principle, as necessary for your design question

6. NGOMSL - why? • More powerful than KLM. Much more useful for analyzing large systems • More built-in cognitive theory • Provides predictions of operator sequence, execution time, and time to learn the methods

7. NGOMSL - Overall Approach • Step 1: Perform goal/subgoal decomposition • Step 2: Develop a method to accomplish each goal • List the actions/steps the user has to do (at as general and high-level as possible for the current level of analysis) • Identify similar methods/collapse where appropriate • Step 3: Add flow of control (decides) • Step 4: Add verifies • Step 5: Add perceptuals, etc. • Step 6: Add mentals for retrieves, forgets, recalls • Step 7: Add times for each step • Step 8: Calculate total time

8. NGOMSL - Example • Car clock • Presetting radio stations • simple • perverse

9. NGOMSL--Car Radio example (1) Provides predictions of methods and operators used to complete a task. If you provide estimates of operator-duration, you can get predictions of error-free expert performance time.

10. NGOMSL-- Car Radio example (2) • Goal/subgoal hierarchy

11. NGOMSL-- Car Radio example (3) • Assumptions • User's hands are NOT on the buttons at the beginning. • Time to move hand & arm to time change level is estimated by 2ft "reach" from Barnes (1963): 410 msec. • D = device time, time for clock to move forward one number, = 500 msec (we estimate this on the next slide) • I am assuming a 12-hr clock. Seems good assumption for a car clock. • Radio is off at beginning and end. • Begin knowing the time you want to set the clock to.

12. NGOMSL-- Car Radio example (4) • D = device time, time for clock to move forward one number, = 500 msec

13. NGOMSL-- Car Radio example (2) • (Duplicate slide) • Note that method for accomplishing the top level goal will have three subgoals and one “release” physical operator

14. NGOMSL-- Car Radio example (5) Need methods for each of these three subgoals

15. NGOMSL-- Car Radio example (6)

16. NGOMSL-- Car Radio example (7)

17. NGOMSL-- Car Radio example (8)

18. NGOMSL-- PreSetting a station (1)

19. NGOMSL-- PreSetting a station (2) Note differences between this and previous slide.

20. NGOMSL-- PreSetting a station (3)

21. NGOMSL-- PreSetting a station (4)

22. NGOMSL-- PreSetting perverse (1)

23. NGOMSL-- PreSetting perverse (2)

24. NGOMSL-- PreSetting perverse (3)

25. NGOMSL-- PreSetting perverse (4)

26. NGOMSL-- PreSetting perverse (5)

27. NGOMSL--comparison of optimal and perverse designs

28. Cost-of-Knowledge Characteristic Function • Card, S. K., Pirolli, P., & Mackinlay, J. D. (1994). The cost-of-knowledge characteristic function: Display evaluation for direct-walk dynamic information visualizations. In B. Adelson, S. Dumais, & J. Olson (Eds.), ACM CHI'94 Conference on Human Factors in Computing Systems (Vol. 1, pp. 238-244). New York: ACM Press.

33. But, • Card, Pirolli, & MacKinley did not do a complete GOMS analysis • Only did enough of an analysis to determine that the “volume part” of the task was the repeated accesses of menus for each component (century, decade, year, month, week, day)

34. Cost-of-Knowledge Function • Small amounts of knowledge can be accessed quickly with access costs increasing as amount of knowledge accessed increases • ≈6.9 s for 1 day • ≈10.4 s for 7 days • ≈14 s for 30 days • ≈17.5 s for 365 days • ≈21 s for 3,562 days • ≈24.6 s for 36,525 days

35. C = Eliminate need to select “week” can do that when select day from the month calendar D = Eliminate “week” & improve cycle time A = Spiral Calendar as is B = SC w/ 2 s cycle time (was 3.5)

36. Design Improvements? • Speed up system response time (SRT) • Time to access = 3.3 + 3.5 * Ncycles • Imply in text that it takes 1 s to point to item and 2.5 s for system to bring up item • Eliminate some of the cycles -- seem redundant • Get rid of the week cycle • Have users bring up month and directly select day from the month calendar rather than first selecting week and then month • Issue -- • Implicit estimate of 2.5 s srt seems tediously long • Suggest need for a full NGOMSL analysis

37. NGOMSL Methods for Spiral Calendar Top method and one subordinate

38. NGOMSL Methods (cont’)

40. How well does NGOMSL fit the data? Pretty well by r2

41. How well does NGOMSL fit the data? Pretty well compared to the equation they derived from the data

42. How well does NGOMSL fit the data? But not too well when I assume a 2.5 sec system response time

43. What Happened? • Did the modeler cheat? • Let’s examine the assumptions of the model • Critical assumptions concern the time per cycle and the elements that compose the cycle • Card et al., empirically derived 3.5 s per cycle • The NGOMSL model, which assumed a 1 s SRT, predicted 3.1 s per cycle

44. 3.1 s 3.1 s 3.1 s

45. Where is the problem? • For NGOMSL to accept 2.5 s SRT means either • accepting a very bad fit, or • changing the model • But, what can be changed? • My guess is that they never measured the SRT of their system • In any event it is incredible that they assumed that all of the cycle time not devoted to pointing was attributable to SRT

46. Two Morals to this Story? • First • Even good researchers can be careless when they do not check their assumptions against a model • (& yes, Card, Pirolli, and Mackinlay are VERY good researchers) • Second • Before investing lots of effort into jazzing up system response time, do a profile analysis with a complete GOMS model

47. Next Week -- TUESDAY!! • NGOMSL#1 • Produce NGOMSL methods from the trace of the VCR (my trace not yours) • Are no pops in NGOMSL. Verifies may be suitable replacements. Yours to decide. • NGOMSL#2 • Create NGOMSL methods for the ManTel task • Should be one set of methods that can be applied to both interfaces and to 1 or 2 queries • Obviously, not all methods need to be used in each analysis • Produce an NGOMSL trace for Dialog-Box 1 query and PopUp 2 queries • Readings • Paul Green’s 1999 Human Factors Ergonomics Society conference article

48. End of NGOMSL