Usability Testing for Usable Accessibility

1. Usability Testing for Usable Accessibility U Kate Walser Whitney Quesenbery

2. Hi Kate Interaction design, usability testing and slipping accessibility under the radar. Whitney Users’ stories, plain language, discovered accessibility as a right in elections What about you?

3. What do you want to learn today? What do you think usability testing is? Does anyone in your organization do any sort of usability testing? What stands in the way of doing usability?

4. Here’s what we planned Introduction to usability testing Demo of a usability test Planning a usability test Take a deep breath First practice test Debrief and moderating skills Break Testing with people with disabilities Practice tests Debrief and applying what you learned

5. Schedule

6. What is usability (testing)? 1

7. Usability is about people

8. Different types ofpeople…doing differenttasks… have differentusability needsdepending on their goals and tasks For this person efficiency and effectiveness are the most important dimensions. For this person, easy to learn and error tolerant are the most important dimensions.

9. Helpful models for thinking about usability A’s Ability, aptitude, attitude (what users bring to the site) E’s Efficient, effective, engaging, error-tolerant, easy to learn (dimensions of usability) Use Useful, usable (operable), desirable (how we define success)

10. Usability testing lets you see behavior Observe • People like the audience using the site rather than asking them about it • How easily they meet their goals • What causes confusion or problems Inform design • Lets you confirm or challenge assumptions about the design. • Gives you the opportunity to fix problems. Informal usability testing of voting materials at the Farmer's Market in Olympia, Washington

11. How easily can we learn from users? We learn from users by watching them use the site or other materials. • Don’t explain or demo. • Watch what they do. • Listen to their comments. • Take their problems seriously. Dona Vitale conducting a usability test in Chicago, 2005

12. A usability test lab A formal lab includes a room where the participants work, and a separate room for observers to watch. Sessions are often recorded on video Michigan State University Usability & Accessibility Centerhttp://usability.msu.edu/default.asp

13. Usability testing with people with disabilities How will a person with disabilities use the product? • Assistive technology use and settings • Other resources they use How usable is the product by people with disabilities? • Content, forms without surrounding visual context, proximity • Navigation • Truly accessible vs. just meets the standards How engaging and effective is your content strategy? • Images only vs. engaging context + images • Search engine readiness (headings, link labels) • Great content flow

14. Demo 2

15. Your goal You want to a group from Access to go fishing at a local park. Find a park that has a wheelchair accessible fishing pier. What are the admission fees?

16. Planning a usability test 3

17. Begin with your goals Your site visitors' goals (and their reality) Your organization'sgoals • Answer people's questions about your topic • Have people complete a transaction correctly without calling • Get more people to fill out a form correctly without calling Success − for youand your users

18. The purpose of the test influences the plan Location and context: formal informal Recruiting: defined opportunistic Activities: instructed tasks free tasks Questions: structured unstructured Data collection: observation task/data only Results quantitative qualitative

19. Think about what senses are most involved in the UX

20. What you need for a usability test What • The material you want to test • Scenarios, if it’s not just an exploratory “try it out” type test Where • A quiet room (Maybe. We know of very successful usability testing in an open marketplace, a hangar at an air show, the chemotherapy center at the NIHClinical Center.) Who • Moderator • Observer/note-taker • Users: 3 − 6 people, one at a time

21. The moderator runs the session • Impartial, unbiased, observing • No teaching! • Listen and watch • Open-ended questions: Why?How? What were you doing? Moderator roles: Flight attendant: Ensuring safety and comfort Scientist: Planning, maintaining objectivity, managing data

22. Observers and note-takers Watch (and listen) quietly for whether the user • Is confused by anything, or has to re-read information • Has to correct mistakes, or backtrack to correct naviation Take good notes • Write down what you see and hear. Don’t “translate” or put words in the user’s mouth • Be specific. Not "he's confused." But "he said he doesn't know what APR means.” • Don't infer the user's reasons for doing something. • Don't solve problems while taking notes. That's for later. It will take all your concentration to note what is happening.

23. Briefing the user • Thanks for trying out this […]. Your doing this will help improve this material. • Note: If you never use the words "test" or "evaluate" with the user,you will not have to say "We are not testing you." • You can stop anytime. • Your involvement will be confidential. • If you get stuck or confused, say so. • Please let me know what you are thinking as you use this …

24. Different ways of having people try out your site Ask participants for their own scenarios. • Have you used this site? What for? Please show me how you did that with this site. or Would you be likely to use this site? What for? Please show me how you would do that with this site. Give participants tasks you have written. • Write scenarios that will have people use parts of the site you are worried about. Customize scenarios for the participants. • Write general scenarios, but adapt them to what you know about the participant, letting them fill in details to make the task more relevant. Use both of these with their own scenario first.

25. Use think aloud during − or after the task Consider asking people to “think aloud” as they work • What they’re doing • Why they’re doing it Assistive technology may make think aloud harder. If so, you can go back over the material (“retrospective”) • Ask the participant to walk you through what they did and why. • Use the material as a guide for the discussion. • Ask if anything was confusing or frustrating.

26. Use simple techniques to help users keep going If the user says, “hmmm” or “oops” or “I wonder…” • Say, “What questions do you have right now?” If you are doing "think aloud" and the user is silent for 10 or 20 seconds (count!) • Say, “What are you thinking?” If users stop because they think they're done or they are stuck (and you think there’s a problem) • Summarize what you saw the user do. • Ask "What would you do now?"

27. How easily can we learn from users? Usability testingdoes not have to be formal, lengthy, or expensive. You don’t need • a formal laboratory • 100s of participants • special equipment (except for your voting system) • special recording systems Poster from Washington State

28. Summary: When to do usability tests Early Understand users and their goals Middle Evaluate the site throughout the development process End Evaluate the finished site to measure its success Ongoing Watch site usage and technical support for evidence of problems Project Initiation Design and Development Product Release In Use

29. Take any opportunity for a quick usability observation The next time someone asks you a question about your site, ask them to use it instead. Watch as they try to find the information.

30. First practice test25 minutes 4

31. Plan your practice test Pick something you want people to do. Review the script for briefing the participant. Write the task instructions. Starting point (desktop, a URL?) Phrase it in their language What do you want to watch for? Remember to thank the participant at the end

32. Ready? Three rounds Get into a group of three. Take turns, so everyone tries each role. We’ll call time.

33. How did that go? What felt awkward? What worked well? Anything surprise you?

34. Moderating skills 5

35. From earlier: The moderator runs the session • Impartial, unbiased, observing • No teaching! • Listen and watch • Open-ended questions: Why?How? What were you doing? Moderator roles Flight attendant Ensuring safety and comfort ScientistPlanning, maintaining objectivity, managing data

36. Moderator Responsibilities Often, but not always, create the test plan and testing scenarios Welcome and support the participant • Informative but not revealing Avoid accidentally influencing results • Explain the objectives succinctly • Helpful but not too helpful • Try not to answer questions that affect outcomes Observe quietly and avoid distracting participants • Thoughtful about taking notes • Ask questions only when they realize participant is at a stopping point or is ready to talk

37. Managing the customer Often, moderators also manage the customer team whose products will be used in the sessions.

38. Prepare the customer team Define and prioritize objectives • Address these in scenarios, probe questions, questionnaire • Determine which are “must haves” and which are optional Describe to the customer the process, logistics • Where they will sit (if they will attend) • What you expect from them (no interruptions, other room, etc.) Explain how you will handle “edge cases” • Late or no-show participants • Technology woes • Questions that arise during the sessions (and how to get moderator’s attention to ask)

39. A good moderator uses a test plan Creates a test plan to use during the session • Manage the usability testing like a mini-project • Provide clear schedule with tasks and deadlines – moderator, customer, etc. • Follow up to make sure everything’s set Creating a test plan and schedule for usability testing helps avoid mistakes and oversights.

40. A good moderator Prepares • Create a test script • Practice on real participants • Verify that everything is ready and there are plan B’s Arrive early – “on time” is “late” Listen and watch with an open mind • If you wonder, ask without bias – don’t jump to conclusions • Story – boy with the black crayons Respect the participant • Watch for “trouble” indicators • Stop as needed due to time, tech issues, or other factors

41. Expect most participants to be… Hesitant or nervous • Are thier skills being tested? • What if they don’t give the “right” answer? Ready to throw their friends and colleagues under a bus before themselves! • Give them an easy way to provide their feedback without worrying about hurting feelings • Ask what their friends or colleagues will think Photo credit: “accuse,” Grey Worldhttp://www.flickr.com/photos/greyworld/ Smiling man points to friend who is covering his face.

42. Great moderators make participants feel valued and important If possible, confirm the session yourself • Call or email the participant to confirm, introduce themselves • Make sure participant knows where to go • Give participant a friendly face to expect Memorize the script Learn negotiation tactics • Maintain an open mind • Manage challenges that arise (Handle surprises without disruption A little girl sitting in dentist’s chair is smiling from ear to ear after learning she has no cavities. Photo credit: “No Cavities!” daveparkerhttp://www.flickr.com/photos/daveparker/

43. Have a backup plan Photo credit: “Usability Testing our Paper Prototypes” Jason de Runahttp://www.flickr.com/photos/jderuna/ Person uses paper printout of screens to describe interaction and path

44. Summary: Moderators Prepare and anticipate problems Observe without influencing outcomes Make participants feel valued

45. Break15 minutes *

46. Practice tests45minutes 6

47. Moderating and observing • Conduct sessions the same way. • Ask the participant to “pause” the AT or lower the volume as they may have thoughts they want to share. • Remember that the AT becomes a large component of the UX • Avoid interrupting the user as they use the AT • Remember to watch the cursor, not the mouse.

48. Get into teams If you are the moderator, use your practice test. If you are an observer, think about What you will watch for Where to sit so you can see, but won’t distract the participant. Challenges observing this person’s AT or accessibility setup. Remember: focus on what happens, not analysis or recommendations

49. Planning a test with people with disabilities15 minutes 7

50. How did that go? What felt awkward? What worked well? Anything surprise you?