Testing and Modeling Users
E N D
Presentation Transcript
Testing and Modeling Users Kristina Winbladh & Ramzi Nasr
Usability • Usability means that the people who use the product can do so quickly and easily to accomplish their own tasks • We want to build usable products • How do we know whether our product is usable? • Testing
Usability Testing • Planning a usability test • Defining goals and concerns • Choosing participants • Design • Predictive Models
Planning a Usability Test • How long does usability testing take? • Important for clients and managers to know • Development does not have to stop • Time factors • Complexity of the product? • How much of the product is to be tested? • Number of participants? • Test preparation? • Test evaluation/report?
Examples • Microsoft • 8-12 weeks • Testers work together with developers • Formal reports • General Electric Information Services • 4-6 weeks • 3 days to determine participants and tasks • Week 1&2 test: scenario & questionnaire design • End of week 2: pilot test • Week 3&4: conducting the test w 12 participants • Week 5: test data analysis • Week 6: finished the report
Can you do it faster? • 1 Week • In-house usability specialists conduct test • Each test covers only a few features • Participants background is already known • Tasks are short • Generate short report • 1.5 days… • Special case
What can you do to make testing successful? • Planning! • Starting testing w/o thinking about: • What aspects might not be as usable as they should be? • How well do the participants represent the actual users? • What tasks will we ask them to do in the short time they have? • What information are we going to collect? • How are we going to analyze the information? • What are we going to do with the analyzed information?
The steps • Define the goals and concerns that are driving the test • Decide who should be participants • Recruit participants • Select and organize tasks to test • Create task scenarios • Decide how to measure usability • Prepare other materials • Prepare testing environment • Prepare test team • Conduct pilot test
Team Work • Usability specialists • Designers and developers • Technical communicators • Trainers • Marketing • Helpdesk or customer assistance
Usability Testing • Planning a usability test • Defining goals and concerns • Choosing participants • Design • Predictive Models
Defining Goals and Concerns • You cannot do exhaustive testing • Make choices among goals and concerns • Move from general concerns to specific ones • Understand sources of goals and concerns
Making Choices Among Goals and Concerns • Clearly define the concern that the study will answer • Prioritize all the concerns
Moving From General Concerns to Specific Ones • Refinement • Example • General: how easy is it to use the system? • Specific: how long does this specific task take?
Understanding Source of Goals and Concerns • Sources: • Task analysis and quantitative usability goals • Timely issues • A heuristic analysis • Previous tests of this or other products
Usability Testing • Planning a usability test • Defining goals and concerns • Choosing participants • Design • Predictive Models
Choosing Participants • Steps for choosing participants: • Developing user profiles • Selecting subgroups for a test • Defining and quantifying characteristics for each subgroup • Deciding how many people to include in a test
Developing User Profiles • Collaborative work with: • Marketing, usability specialists, product designers • Use • Surveys, focus groups sessions, contextual interviews, current users of previous version • Characteristics: • Those that all users will share • Those that might make a difference among the users
Profile factors • Work experience • General computer experience • Specific computer experience • Experience with this product • Experience with similar products
Selecting Subgroups for a Test • Dividing users into subgroups • Adding more characteristics • Selecting the most critical characteristics • Gathering other information
Defining and Quantifying Characteristics of Each Subgroup • “Intermediate” users • Setting a minimum and a maximum for a subgroup • Selecting a range of participants within each subgroup
Deciding How Many People to Include in a Test • Less than half of usability problems found with 3 participants (Nielson and Molich, 1990) • 80% of usability problems detected with 4-5 participants (Virzi, 1992) • 90% of usability problems detected with 10 participants (Virzi, 1992)
Deciding How Many People to Include in a Test (Cont.) • The number of participants to include in a test depends on: • Number of subgroups needed • Budget and time • Importance of computing statistical significance of results
Usability Testing • Planning a usability test • Defining goals and concerns • Choosing participants • Design • Predictive Models
Design • Techniques • Video, interaction logging, questionnaire, interviews • Tasks • Time to complete • Number/Type of errors per task/unit time • Number of help requests • Number of users per error/success • Testing conditions
Design (Cont.) • Variables and Conditions • Allocation of participants • Different participants • Same participants • Matched pairs of participants • Data collection and analysis
Usability Testing • Planning a usability test • Defining goals and concerns • Choosing participants • Design • Predictive Models
Predictive Models • Measure user performance without testing the users • GOMS (Goal, Operators, Methods, Selection rules) • The Keystroke Level Model
