1 / 18

Larson 6

Larson 6. Usability Test. Test Plan. Purpose Problem statement/test objectives Subject profile Test design Monitor Evaluation measures (data to be collected) Report contents and presentation. 1. Purpose. Which type of test? Comparison test—Which is “better?”

mmeissner
Télécharger la présentation

Larson 6

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Larson 6 Usability Test

  2. Test Plan • Purpose • Problem statement/test objectives • Subject profile • Test design • Monitor • Evaluation measures (data to be collected) • Report contents and presentation

  3. 1. Purpose • Which type of test? • Comparison test—Which is “better?” • Pilot test—Test the test procedure • Field test—Can real users really benefit by using the system • Beta test—Users help debug the system • Acceptance test—Validate that the system satisfy the requirements? • What do we hope to learn?

  4. 2. Test objectives • Avoid unfocused and vague problem statements • Examples of poor objectives • Is the current product usable? • Is the product ready for release or does it need more work? • Examples of good objectives • Do the screens reflect the content model? • Is help easier to access via a “hot key” or via a mouse selection?

  5. 3. Subject profile • General computer experience • Range: none to two years • Education: • Range: 10% high school, 60 % college, 20% masters 10% Phd • Age • Range: 85% ages 20-50, 15% other • Gender • Education Major: 0% CS, 100% other

  6. 4. Test design • Detailed plan for conducting the test • Groups of subjects, e.g., • Group A: Mary, Fred, Sam, Jose • Group B: Sue, Ron, Bob, Sally • Tests • Group A does test 1 • Group B does test 2 • Subjects read from printed instruction script

  7. Instruction script • Orientation • Introduce yourself • Offer refreshments • Explain why subjects are here • To test a new application • Stress that the subject is not being tested • Describe the equipment the subject will use • Explain that session is being recorded • Nondisclosure form • Lists of tasks to perform. For each task: • Provide data user needs to perform the task • Do not explain what commands to widgets the user should use. • Written debriefing questionnaire

  8. In class exercise • Write Test Instruction Script for your application

  9. 5. Monitor the test • Try to be objective • Enable rather than lead the subject • Don’t act too knowledgeable • Don’t jump to conclusions • Let the subject struggle

  10. 6. Evaluation measures • Performance • Data collected during the test • Preference • Opinions collected during debreafing

  11. Two types of usability tests Is the application useful? Is the application enjoyable? Performance Preference Measure users’ likes and dislikes Measure what the user actually accomplished Validate that users enjoyed the application and will uses it again Validate that users achieved success

  12. Caller Task Measure Typical Criteria Locate “Modern Girl” by Sleater-Kinney Complete successfully Less than 40 seconds Purchase “Modern Girl” Complete successfully Less than 2 minutes Performance Metrics

  13. Caller Task Measure Typical Criteria Locate “Modern Girl” by Sleater-Kinney Complete successfully Less than 40 seconds Purchase “Modern Girl” Complete successfully Less than 2 minutes Performance Metrics Write performance metrics for your application

  14. Question Typical Criteria On a scale from 1 to 10, rate the help facility. The average caller score is greater than 7. On a scale from 1 to 10, rate the understandability of the error messages The average caller score is greater than 7. Would you recommend using this applications to a friend? Over 75% of callers respond by saying “yes.” What would you be willing to pay to use this voice-enabled application? Over 75% of callers indicate that they are willing to pay $1.00 or more per use. Preference Metrics

  15. Question Typical Criteria On a scale from 1 to 10, rate the help facility. The average caller score is greater than 7. On a scale from 1 to 10, rate the understandability of the synthesized voice. The average caller score is greater than 7. Would you recommend using this applications to a friend? Over 75% of callers respond by saying “yes.” What would you be willing to pay to use this voice-enabled application? Over 75% of callers indicate that they are willing to pay $1.00 or more per use. Preference Metrics Write preference metrics for your application

  16. Preference Metrics (Open-ended Questions) • What did you like the best about the voice-enabled application? Do not change these features • What did you like the least about the voice-enabled application? Consider changing these features • What new features would you like to have added? Consider adding these features in this or a later release. • What features do you think you will never use? Consider deleting these features. • Do you have any other comments and suggestions? Pay attention to these responses. Callers frequently suggest very useful ideas.

  17. Write open-ended questions for your application

  18. Lab Assignment • Write a text plan • Write the usability test design • Conduct the usability test • Record test data and summarize each of the performance and preference metrics • Place all of your documentation into your lab log and send URL to instructor

More Related