Instructional Technology & Usability

Instructional Technology & Usability MATTERS Leslie Carter, Ph.D. Usability Testing for ISD - Leslie Carter

Instructional Technology & Usability 1. How important is usability? 2. What do we mean by “usability?” 3. How do we evaluate usability?

According to Instructional Designers • 99% Agree: Usability is essential to e-Learning design. • 96% Agree: e-Learning components should always be tested for usability. • 96% Agree: User-centered design enhances the effectiveness of e-Learning. From the E-Learning Guild’s “Usability and e-Learning” survey reported 2/27/2004. Usability Testing for ISD - Leslie Carter

According to Instructional Designers What are the "Top 5" features you would recommend that someone consider when licensing a live e-learning product? Ease of Use was cited by more e-learning professionals than any other factor. From Brandon-Hall.com winter 2004 survey. Usability Testing for ISD - Leslie Carter

Instructional Technology & Usability 1. How important is usability? 2. What do we mean by “usability?” 3. How do we test for usability?

Usability is “The effectiveness, efficiency, and satisfaction with which specified usersachievespecified goalsinparticular environments.” (ISO 9241) 1. Effectiveness - Accuracy & completeness. 2. Efficiency – Speed & ease of use. 3. Satisfaction - Comfort & acceptability. Usability Testing for ISD - Leslie Carter

What contributes to usability? A. The overall process of designing how a user will interact with a software application or web site, including: requirements analysis, information architecture, interaction design, visual design, user testing, documentation, & help system design. B. The design of the graphic layout, & the images, icons, fonts selected to support the informational content. C. Information organization that should help people find & use information efficiently. It’s concerned with things like content chunking, formatting, categorizing, labeling, placement & navigation. D. The design of the workflow through an application or site, including the dialog between the user & the system, what user inputs are needed & when, as well as, what feedback is appropriate & when. E. The design and placement of menus & options used to find specific areas of a software application or site. • Visual Design • Information Architecture • Navigation Design • Interaction Design • User Interface Design Usability Testing for ISD - Leslie Carter

Guidelines for Good Usability Review the usability heuristics (guidelines) published by researchers & practitioners. Learning these will help you: • Recognize the cause of problems found in usability testing • Find solutions to those problems • Reduce the number problems found in future usability testing A Comparison of Heuristics from Several “Gurus” Usability Testing for ISD - Leslie Carter

Role of Usability in Evaluation Poorusability leads to: • Increased cognitive load • Wasted time • Frustration & reduced motivation Great training content can lead to lousy training results when poor usability is involved! Usability Testing for ISD - Leslie Carter

Instructional Technology & Usability 1. What does the beginning of the field of usability have in common with instructional technology? 2. What do we mean by “usability?” 3. How important is usability? 4. How do we test for usability?

Conducting a Usability Test • Determine goal of test • Select evaluation criteria • Select participants • Write a task scenario • Write a testing protocol • Conduct the test • Track the observations • Report the results Usability Testing for ISD - Leslie Carter

Testing Goals • Determine how to improve a developing product • Formative: product under construction, could be even be a prototype • Informal: quantitative measure not required, qualitative observations needed • Compare two different products or versions of same product • Summative: products are mature & ready for the marketplace • Formal: quantitative measures required Usability Testing for ISD - Leslie Carter

Evaluation Criteria • Most common • No. of tasks completed • Time to complete task • Errors made in performance • Satisfaction • Other possible measures • Proportion of time spent determining what to do instead of doing it • No. of times consulted help or user guide • No. of positive or negative comments from user Usability Testing for ISD - Leslie Carter

Usability is “The effectiveness, efficiency, and satisfaction with which specified usersachievespecified goalsinparticular environments.”(ISO 9241) 1. Effectiveness - The accuracy & completeness. 2. Efficiency – Speed & ease of use. 3. Satisfaction - Comfort & acceptability. Usability Testing for ISD - Leslie Carter

Participants • Similar to your audience – training, experience… • Availability – when & for how long. • Where – your place or theirs, theirs is better if like that of your target audience. Consider remote testing if you can’t get to them. Usability Testing for ISD - Leslie Carter

Tasks Scenario • Select tasks: critical &/or frequent • Develop a realistic scenario for sequencing those tasks • Start with easy tasks to build user’s confidence • For each task give user a goal to achieve instead of steps to perform • Just enough info for user to complete the task without more info from observers • Make it clear to user when each task is compete Usability Testing for ISD - Leslie Carter

Sample Scenario Outline Task 1 - View a webshop Task 2 -Create a webshop Task 3 -Revise a webshop Task 4 -Create a course (several webshops put together) Task 5 - Revise a course Usability Testing for ISD - Leslie Carter

Sample Tasks Task 1 – View a Webshop Stating from the T/TAC home page, find “Leslie’s Webshop” & view the entire webshop. Task 2 - Create a Webshop Your goal for our first task is to create a simple webshop made up of several kinds of media. Name this webshop with your name and the date, e.g., “Leslie’s 3/1/04 Webshop.” This webshop will include the sample media we brought & consist of 2 pages: • A page with one large graphic (Image-1large.jpg). Title this page “My First Page.” Add this description “my webshop.” Check to see how this page will look to the site’s viewers before starting on the next one. • A page with a video clip (TinyMovie3.mov), and an Acrobat PDF file (acrobat6matrix.pdf). Title this page “My Second Page.” Usability Testing for ISD - Leslie Carter

Testing Protocol • Intro for participants • Reminders to observers • Copy of the task scenario • Steps required to perform each task • Make a separate copy for each observer & for each test Usability Testing for ISD - Leslie Carter

Sample Protocol – Participant Intro • We are not testing your ability to use this site; we are testing the sites’ ability to meet your needs. • Your feedback will help us ensure that T/TAC Online will meet the needs of T/TAC professionals. • The site is designed to allow T/TAC staff to upload training content for presentation over the Internet. There are two kinds of presentations we’ll try today. One is a webshop which is a presentation designed to last 15-30 minutes, something a viewer might complete in one sitting. The other is a course which is a set of webshops combined in one course. • If you would like to take a break at anytime, just let us know. • Please try to think out loud as you use the site. Let us hear the questions in your mind. We need to know if you can’t find something or if something is not absolutely clear to you. And, let us hear about the features you like about the site. Usability Testing for ISD - Leslie Carter

Sample Protocol – Conducting the Test • Reminders: • Don’t tell user the steps, unless they’re very frustrated • Don’t give away the names of the buttons or links • Note start & end times, step where error occurs • Get positive & negative quotes from user Usability Testing for ISD - Leslie Carter

Tracking Results • It’s very easy to forget or loose site of useful observations • Need a system for tracking all the issues you want to address • Track who noticed the issue, when noticed, what kind of issue, what screen, how important, & when fixed. Usability Testing for ISD - Leslie Carter

Reporting on a Test • MUCH stronger impression is communicated if you have sample user comments from videotapes or Camtasia screen recordings. • There is an industry standard for usability test reporting from NIST. Very complete. The checklist is great. http://zing.ncsl.nist.gov/iusr/documents/cifv1.1b.htm Usability Testing for ISD - Leslie Carter

Usability Testing in Kuniavsky • One day before a test is not enough for the practice test. (p.264) • Excellent discussion of dealing with participant recruiters (p.266) • Don’t test features. Test user goals. (p.268) • Quantitative measures – what are the goals of your test? Don’t forget organizational politics. Usability Testing for ISD - Leslie Carter

Usability Testing in Kuniavsky (cont.) • Make it absolutely clear that it’s the interfaces, not the person being evaluated. (p.279) • Discount usability testing – Don’t need one-way mirror or cameras. You can do it with remote software & screen capture utilities. (p.286) • Define severity ratings to match the bug codes & enter them in the bug tracking system. (p.294) Usability Testing for ISD - Leslie Carter

Resources http://immersion.gmu.edu/portfolios/lcarter3/resourcesusab.html Usability Testing for ISD - Leslie Carter

Instructional Technology & Usability