Usability Engineering and its role in Software Industry

1. Usability Engineering and its role in Software Industry Qaiser S. Durrani FAST-NU, Lahore Workshop on Usability Engineering Feb 21-23, 2011 at SEECS NUST

2. Agenda • Usability Engineering? • Why we need it? • What are its measures? • Where UE fits in the SDLC • Can we integrate or map UELC with SDLC? • Experience and Emotional Measures – Role? • Case Study • Current practices in Software Industry with respect to UE

3. Why Usability Engineering? • Functional perspective • User perspective

4. Usability • ‘‘the capability to be used by humans easily and effectively’’ • ‘‘quality in use’’ • ‘‘the effectiveness, efficiency, and satisfaction with which specified users can achieve goals in particular environments’’ • Context dependent (shaped by the interaction between tools, problems, peoples) • A process through which usability characteristics are specified and measured throughout the software development lifecycle.

5. Key Research Questions in HCI • How to work with and improve the usability of interactive systems? • Guidelines for improving the usability of systems? • Methods for predicting usability problems? • Techniques to test the usability of systems? • Discussions on how to measure usability

6. Neglecting Usability Engineering

7. Usability into Software Development • When integrating usability into the system design process, early focus on users and tasks, empirical measurement, and iterative design principles are suggested • This integration, however, is not a trivial task, as numerous obstacles have been reported • First of all, introducing a new method into a software development organization is typically a delicate problem • User-centered design techniques have been reported to remain the speciality of visionaries, isolated usability departments, enlightened software practitioners, and large organizations, rather than the everyday practice of software developers

8. Usability Engineering and Experience Design

9. Models for Usability Engineering Lifecycle • Star Lifecycle Model • ISO 13407 Model • Usability engineering lifecycle by Deborah J. Mayhew

10. Usability Engineering Lifecycle

11. Requirements Analysis Phase • User Profiling – Cognitive & Non-Cognitive measures • Task Analysis • SW/HW/Environment Constraints • General Design Principals • Usability Goals

12. Design, Development & Evaluation • Conceptual Level Design • Detail Level Design • Screen Standards • Iterative Evaluation

13. Usability Activities

14. Adaptation of Usability Activities into Software Engineering Development Process

15. Allocation of Usability Techniques toDevelopment Activities

16. Shneiderman’s Golden Rules • R1:Strive for consistency • R2:Offer shortcut • R3:Give effective feedback • R4:Reduce Short term memory load • R5:Provide reversal of actions • R6:Design Dialogues to yield closure • R7:Provide locus of control

17. Practices - MEASURING USABILITY(Case study of 180 projects) Measures of effectiveness Measures of Efficiency Measures of Satisfaction

18. Measures of Effectiveness • Binary task completion • Accuracy • Recall • Completeness • Quality of outcome • Experts assessment

20. Comments 1- 22% of the studies reviewed do not report any measure of effectiveness nor do these studies control effectiveness. Frøkjær et al. argued that the HCI community might not succeed in trying to make better computing systems without employing measures of effectiveness in all studies 2- Research shows that measures of the quality of the outcome of the interaction are used in only 16% of the studies. For example, experts’ assessmentof work products seems a solid method for judging the outcome of interaction with computers and has been used in a variety of fields as an indicator of the quality of work products, for example with respect to creativity. Yet, in this sample only 4% of the studies use such measures

21. Comments 3-New kinds of devices and use contexts require new measures of usability. Especially, it has been argued that the notion of task underlying any effectiveness measure will not work in emerging focuses for HCI, such as home technology 4- A number of studies combine usability measures into a single measure, report the combined values, and make statistical tests on the combinations

22. Measures of efficiency • Time • Input rate • Mental effort • Usage patterns • Communication effort • Learning

24. Comments 1- Some of the efficiency measures are obviously related to the quality of interactive computer systems, because they quantify resources (e.g., time or mental effort) that are relevant in many contexts for many users 2- A second comment on the studies reviewed pertains to the measurement of time. A surprising pattern apparent from Table is that while objective task completion time is measured by 57% of the studies, little attention is paid to user’s experience of time • However, in this sample of 180 studies, only one study measures directly subjective experience of time

25. Comments 3- The reviewed studies differ in how task completion times, and efficiency measures in general, are reasoned about. In the ISO definition of usability and in most of the studies reviewed, time is considered a resource of which successful interfaces minimize consumption • However, in a handful of studies higher task completion times are considered as indicators of motivation, reflection, and engagement

26. Comments 4- A striking pattern among the studies reviewed is that few studies (5) concern learning of the interface. • Only five studies measure changes in efficiency over time 5- In the studies reviewed, the median time of working with the user interfaces evaluated was 30 min

27. Measures of Satisfaction • Standard questionnaires • Preferences • Satisfaction with the interface • User attitudes and perceptions

30. Comments 1- The measurement of satisfaction seems in a state of disarray. A host of adjectives and adverbs are used, few studies build upon previous work, and many studies report no or insufficient work on the validity and reliability of the instruments used for obtaining satisfaction measures • Another indication of the disarray is in the limited use of standardized questionnaires

31. Comments 2- A second comment on the satisfaction measures used is that studies vary greatly in the phenomena that are chosen for objective performance measures and those that are investigated by asking subjects about their perceptions and attitudes. • One question arises when users’ perception of phenomena is measured when those phenomena perhaps more fittingly could have been assessed by objective measures 3- The review shows that in practice subjective satisfaction is taken to mean a questionnaire completed after users used the interface. Only eight studies (4%) measure satisfaction during use without using questionnaires

32. CHALLENGES IN MEASURING USABILITY

33. Subjective and objective measures of usability • Measures of usability concern user’s perception of or attitudes towards the interface, called subjective usability measures • Other measures concern aspects of the interaction not dependent on user’s perception called objective usability measures • Such a distinction has been argued to simplify the nature of measurement in science • Suggest using the distinction to reason about how to choose usability measures and find more complete ways of assessing usability • Measures may lead to different conclusions regarding the usability of an interface

34. Measures of learnability and retention • Particularly measures of efficiency, we find it relevant to compare them to recommendations on how to measure usability The well-known textbook by Ben Shneiderman (1998, p.15) recommends measuring (1) time to learn, (2) speed of performance, (3) rate of errors by users, (4) retention over time, and (5) subjective satisfaction. Nielsen (1993, p. 26) similarly recommends measuring (a) learnability, (b) efficiency, (c) memorability, (d) errors, and (e) satisfaction • Most of the reviewed studies follow part of the recommendations by measuring task completion time (points 2 and b above), accuracy (points 3 and d), and satisfaction with the interface (points 5 and e): 92% of the studies measure at least one of these; 13% of the studies measure all three

35. Measures of learnability and retention • The majority of studies make no attempt to measure learnability or retention • This challenge is most relevant for studies or research addressing systems that users should be able to learn quickly or that will be intensively used • Overall, usability studies could put more emphasis on measures of learning, for example by measuring the time needed to reach a certain level of proficiency • In addition, measures of the retention of objects and actions available in the interface (i.e., the ability of users to come back and successfully use the interface) are important in gaining a more complete picture of usability

36. Measures of usability over time • The studies reviewed show that users typically interact only briefly with interfaces under investigation; as mentioned earlier the median duration of users’ interaction was 30 min; only 13 studies examined interaction that lasts longer than five hours • The brief period of interaction in the studies reviewed explains the lack of focus on measures of learning and retention • The observation also suggests that we know little about how usability develops as the user spend more time interacting with the interface and how tradeoffs and relations between usability aspects change over time • From research, we need a more full understanding of how the relation between usability aspects develops over time

37. Extending, validating and standardizing measures ofsatisfaction • The disarray of measures of satisfaction presents special challenges • One is to extend the existing practice of measuring satisfaction almost exclusively by post-use questions; • another is to validate and standardize the questions used • Validation may be achieved through studies of correlation between measures

38. Micro and macro measures of usability • Usability at a micro level • Such measures cover tasks that are usually of short duration (seconds to minutes), has a manageable complexity (most people will get them right), often focus on perceptual or motor aspects (visual scanning, mouse input), and time is usually a critical resource • Usability at a macro level • Such measures cover tasks that are longer (hours, days, months), are cognitively or socially complex (require problem-solving, learning, critical thinking, or collaboration)

39. A working model for usability measures and researchchallenges

40. Affective Requirement • The need to make something fun, engaging, or enjoyable is usually not considered in requirements elicitation • Software requirements for these and other affective factors are never truly captured in an official manner • Juran is credited with coining the phrase "fitness for purpose“ • If a system is intended to be a leisure product then the ‘fitness for purpose’ must also extend to affect

41. Rebirth of Affect in Design • The idea of affect is not old but affect has re-emerged as a potentially desirable design characteristic • One of the visionaries of this re-emergence was Robert Glass from Sun Microsystems, who said: “If you’re still talking about ease of use then you’re behind. It is all about the joy of use. Ease of use has become a given – it’s assumed that your product will work.” (Glass, 1997)

42. Summary of research into affective factors

43. Exploring Affect……Theories • Three theories have each been said to contribute to computer game enjoyment Usability: • In ISO 9241-11 (ISO, 1998), usability is characterized as consisting of three elements: • effectiveness, efficiency, and satisfaction • Grice (2000) attempted to apply these three elements to computer game design • His hypothesis was that computer games that were enjoyable will have high levels of efficiency, effectiveness, and satisfaction • Some minor experiments conducted under his supervision seemed to indicate that this hypothesis was true

44. Exploring Affect…Theories Flow: • Csikszentmihaly describes flow as ‘the holistic sensation that people feel when they act with total involvement • In the state of flow, actions flow without conscious intervention by the actor • The term flow was used because people in this state often said that they “were in the flow of [the activity]”. • the characteristics of flow-inducing activities are: • must feel capable of completing the task • must have the ability to concentrate on task • clearly recognizes the goals of the task • receives immediate feedback about task performance • has a sense of control over their actions • has the sense of time altered: hours can seem like minutes

45. Exploring Affect…Theories Heuristics for internally motivating interfaces: • Malone (1983), in agreement with Csikszentmihaly, believes that fun and enjoyment only arise from activities that are intrinsically motivated • Computer games are thought to be played because of intrinsic motivation, with no expectation of a reward other than the activity itself • Malone and Lepper (1987) developed seven heuristics for the design of intrinsically motivated interfaces

46. Exploring Affect…Theories The 4 major heuristics are: • Challenge- multi-layers of challenge so that the user will feel initial success and continue to see improvements • Curiosity- believe that their knowledge structures (or skills) are incomplete or inconsistent • Control-interface should make the user feel that the outcomes are determined by the users own actions • Fantasy- evoke mental images of physical or social situations Other minor are Competition, Cooperation, Recognition

47. Results • The results being referred to are the learnability and ‘losing time’ reasons • Loss of Time • Learnability

48. Measures of specific attitudes towards the interface (Experience Design) – from 180 projects

50. Current Usability Practices in Pakistan Software Industry