Understanding methods in a ‘system of usability practice’

1. Understanding methods in a ‘system of usability practice’ Dominic Furniss PhD Supervisors: Ann Blandford & Paul Curzon

2. Contents • The problem • Overview of the research process • Introduction to accident models and RE • Details of a ‘system of usability practice’ • Validation • Conclusion and the bigger picture

3. Why I chose the problem and why it’s interesting • We don’t know enough about usability methods in practice… • 1. Many methods do not transfer from research to practice • …we need to know more about our users (practitioners), tasks and their contexts to do better applied research… • 2. Literature on methods is criticised as being irrelevant to practitioners • …we need to know more grounded in practitioner perspectives… • 3. Research is moving out from the technical capability of methods, to theiruse, to their use in practice, to wider factors that impact on the performance ofpractice. • …we need to know how methods integrate with the wider system.

4. Overview of the Research Process • Literature review: composition of usability practice knowledge. PITC. • Grounded theory: exploratory, engages with practitioner perspectives. • Bottom-up mode • Website domain analysis (9 interviews) • Safety domain analysis (13 interviews) • Top-down mode • Distributed Cognition perspective • Resilience Engineering perspective

5. Accident models and RE The alarm for reactor signals Operator turns off alarm The reactor overheats ‘Human error’ Sequential Epidemiological Design – too many alarms Training – poor Procedures – not followed Management – weak Safety culture – weak Latent conditions

6. Accident models and RE Non-sequential Events Come together by Bad luck -Murphy’s Law “This cannot be happening to me” Systemic

7. Accident models and RE Playing the devil’s advocate: 3 reasons why practitioners don’t use methods (Senior HCI lecturer, 2005, personal communication)

8. Accident models and RE Transferring HCI Modelling Techniques and Design Techniques to Practitioners (Buckingham Shum & Hammond, 1994) Organisational factors Consultancy gulf Payback gulf Cost gulf Prerequisite gulf

9. Resilience Engineering account. Highlights: • Methods are coupled with project structures and processes • Methods should provide leverage for understanding the project/domain issues • Methods should support rapport, be persuasive, and lead to the development of the practitioner’s reputation • Methods are coupled with staff development and management • The development of tools, methods, and communication practices affect practice • Methods should support documentation where appropriate

10. FRAM: Overall system

11. FRAM 1: Project process

12. FRAM 2: HF/usability practitioner understanding

13. FRAM 3: Persuasion, rapport and reputation

14. FRAM 4: Staff development and management

15. FRAM 5: Tools, methods and reporting practices

16. FRAM 6: Auditing and documentation

17. FRAM: Overall system

18. Validation • Constant comparative method as part of GT • Internal member checking of quotations • Internal (10) and external (8) member checking of the model • Model vs instantiations • Scoping model in terms of detail and complexity • Some parts are not unique to HF/usability • Differences in tone: facilitating and helping vs persuading and selling • Triangulation • People, projects and domains • Literature; e.g.: DC, RE, and Fitzgerald (1996) in IS development • Corroborating and shaping usability practice research

19. Key Contribution to Knowledge • A systems-thinking perspective • Methods should be understood within a theory of system performance i.e. how they affect, and are affected by, wider aspects of HF/usability practice. • In practice methods are simultaneously coupled to the… • … technical substance of project • … communication of results, issues and reporting • … client resources, budgets, and structure • … capabilities of the practitioners involved • … tools – to enhance and extend capability • … rapport, relationships and reputations • … creation of documentation where appropriate • In practice methods • … should not be thought of as tools merely to find problems • … should not be thought of in terms of solely how they are prescribed A’ A

20. The Bigger Picture

21. FRAM: Overall system

22. Thank-you