Usability testing: A reality check for Plain Talk

1. Usability testing:A reality check for Plain Talk A brief overview October 7, 2008 Dana Howard Botka Manager, Customer Communications, L&I Plain Talk Coordinator, Office of the Governor

2. What is usability testing? Finding out if your targeted customer can actually use your product, such as a: • Set of instructions • Form • Web site • Software application • Automobile • Cell phone

3. How does it work? • 4-6 people try out your document, web page, or other product one a a time. • One facilitator conducts the try-out, asking open-ended questions. • The usability participant is asked to “think out-loud” while doing tasks. • Up to two program reps observe and take notes – or record the session.

4. We don’t ask testers what they like We develop a test to see if the typical “user” can actually do a task, such as: • Calculate a late fee. • Fill out a form without making errors. • Find the answer to a question on a web site. • Correctly figure taxes. • Turn on a car defroster quickly. • Forward a cell phone call.

5. Why usability testing for Plain Talk? Because you discover: • Terms customers don’t understand. • Words that can mean something else. • Basic information or context that is missing. • Information that is unimportant & distracting. • Your customer’s emotional reaction to what you’ve developed. • What you had no idea would happen!

6. How does usability work? • Recruit 4-6 people who fit the description of your typical user – but who are not familiar with the “product.” • Develop 6-10 tasks that, if performed correctly, will let you know they can “use” your product. • Write a scenario that will allow the test participant to understand what situation they are in. • Write a series of questions that will get the test participant to do a task – without hints.

7. L&I’s “Quarterly Report” Project:Helping employers self-report hours Challenge: • High customer error rates on this report. • Errors often lead to late fees for employers and costly processing headaches for L&I. Solution: • Make “paper reporting” easier for customers with instruction sheet and “late fee worksheet.” • Encourage employers to use upgraded online report application.

8. Our test materials See the documents in your packet • Before & after examples of our test document • Pre-test screening questionnaire • Instructions and legal sign-off • Scenario and test questions • Sample “thinking aloud” from a test participant • Findings and Recommendations

9. Form before revision

10. Form instructions before revision

11. Setting up the usability test. Plain Talk team members decide what they most need to learn from recruited test participants.

12. Form after revision

13. Form instructions after revision

14. Usability testing on a budget • Recruit test participants from friends, family, agency walk-ins. • Keep the test simple and short. • Use results as a guide – not final data. • Consider the act of developing the test as a great way to focus on your project goals.

15. What about a survey or focus group? • Not a replacement for a test of whether someone can do a task. • People will often say they “like” a document or web site – and not know how to use it. This is because they don’t want to: • Admit they had trouble. • Hurt your feelings. • Go to the trouble of seeing if it actually “works.”

16. Invite managers and staff to observe testing! It’s a great reality check.

17. Involve frontline staff in usability testing. They know their customers – and are great at solving problems.