1. Nina Reeves, Module Tutor 1 MU220 User Interaction Studies Observational studies for usability evaluation

2. Nina Reeves, Module Tutor 2 Overview of learning outcomes By the end of this session you should be able to: Understand what is meant by ‘empirical studies’ Understand the nature of observational evaluations Understand the issues related to observational evaluations Have carried out an observational study of a game

3. Nina Reeves, Module Tutor 3 Empirical studies Remember definition of “Usability” “… the effectiveness, efficiency and satisfaction with which specified users can achieve specified goals in particular environments.” ISO DIS 9241-11 Effectiveness – to what extent goal/task is achieved Efficiency - effort required to complete the goal/task Satisfaction – level of comfort/acceptability that users feel Is your mobile phone a ‘usable’ multimedia device? In UK/EU but not in the USA? Cost? What about in a Cotswold valley/LC115? Usability is not always to do with the design of the interface, the context is also important Designers derive ‘Scenarios of use’

4. Nina Reeves, Module Tutor 4 Gathering data Observational study Mobile Phone game In group of 6 Observe 1 person playing a game Mobile Phone calculator Task 1 58 x 9 / 1.3 Task 2 (505 + 398) / (23 - 4) + + Mobile phone camera Need 2 participants and observers Pass to someone who doesn’t have a camera phone Owner turns their back on the user and attempts to explain verbally how to take a photo. What kind of usability data can you collect? Quantitative Qualitative

5. Nina Reeves, Module Tutor 5 Possible hazards of observation Taking notes concurrently with observing Need for materials/paper to collect/log/code behaviours Think aloud protocol yields large amount of data Possible solution - Video recording for later review? How do you analyse the notes from all observers? Wanting to offer advice/help Researcher bias – particularly if it’s your “baby” Pressure felt by participant being observed Stress? Hawthorne effect – better performance when observed Lack of well-defined task Calculator task well-defined, game? Dependence on skill level of participant Game/calculator

6. Eye tracking for observation Nina Reeves, Module Tutor 6 http://www.youtube.com/watch?v=v0xG1xQTf9E&feature=related George Clooney Martini advert Overview video http://www.etre.com/usability/eyetracking/showme/ http://www.etre.com/blog/2006/05/five_days_five_heatmaps/ Tobii h/w and analysis s/w http://www.tobii.com/corporate/start.aspx http://www.youtube.com/watch?v=xgRIjrlK1mA evidence from Lancaster Uni research http://www.youtube.com/watch?v=bZOVcmwHZZk&feature=related Heatmaps and eyegaze maps for Google, Flikr photo sharing iPhone eye tracking http://www.youtube.com/watch?v=uQrFbyMcF40&feature=related

7. 15 min Research Ex Eye tracking sounds positive but… Google search for alternative views and list possible problems with this approach Who says that eye tracking is great? Who says that it is not the ultimate technique? Nina Reeves, Module Tutor 7

8. Nina Reeves, Module Tutor 8 Usability investigations (review) 3 Lab pracs run on Weeks 1-3 Week 1 Heuristic evaluation You acted as ‘experts’ to gather data Week 2 Observational study of games Parking – use the LabPrac2.doc to help you Week 3 Focus group In parallel - Assignment planning Allocation of Usability study groups Read the next slide!

9. Nina Reeves, Module Tutor 9 Thinking ahead! Planning – Weeks 5 - 7 'Planning is an unnatural process: it is much more fun to do something. And the nicest thing about not planning is that failure is a complete surprise rather than being preceded by a period of worry and depression.' John Harvey-Jones You need to start planning!

10. Nina Reeves, Module Tutor 10 Practical session overview In a pair Practise setting up an observational study using the Parking game suggested (on lectures page) LabPrac_2_10usability of a game.doc At appropriate time Meet together as Assignment 2 team and exchange contact details – list on website

11. Nina Reeves, Module Tutor 11 Any Questions?

12. Nina Reeves, Module Tutor 12 Bibliography Benyon D, Turner P. & Turner S. (2005) Designing interactive systems: people, activities, contexts, technologies. P.216- 220. Harlow, Pearson Education. Dix A, Finlay J, Abowd G. and Beale R. (2004) Human-Computer Interaction. 3rd ed. New York, NJ, Prentice-Hall. (Ch. 6 & 9) Faulkner X. (2000) Usability engineering. Basingstoke, Macmillan press. (Ch. 6) Jordan P.W. (1998) An introduction to usability. London, Taylor and Francis. Nielsen J. (1993) Usability Engineering. Boston, MA, Academic Press. (Ch.7) Norman D. (1990) The design of everyday things. Cambridge, MA, MIT Press. Preeces J. (1993) A guide to usability. Wokingham, Addison-Wesley. Shneiderman B. (1998) Designing the user interface. 3rd ed. Reading, MA, Addison-Wesley. (Ch. 4) More available from the Resources section of the MU220 website NB: You are NOT doing an Heuristic evaluation for the Assignment 1!

13. Nina Reeves, Module Tutor 13 Bibliography of ISO standards ISO 9241-11:1998 Ergonomic requirements for office work with visual display terminals (VDTs) -- Part 11: Guidance on usability - Guidance on Usability ISO 13407:1999 Human Centred Design Processes for Interactive Systems ISO 14915-1 Design Principles and Framework (2001) FDIS registered for formal approval ISO 14915-2 Multimedia navigation & control (2001) Enquiry stage ISO 14915-3 Media selection and combination (2001) FDIS registered for formal approval