Gathering Data from People

1. Gathering Data from People HCC 729 4/3/14

2. This week • Mini-lectures: Connie, Jonathan • Finalizing P2 ideas • Data collection and research methods

3. Inspirations

4. Comments on reading • Contextual design • Cooperative inquiry

5. Project 2 ideas • Designing applications of future technology • Personal robotics / drones • Wearable computers • Now: finalize general idea • Underlying technology • User / task / environment • Let’s go around and get project updates

6. Today: Data gathering methods • How to understand your user

7. Data Gathering Methods • Methods for personas and Scenarios • Don’t interact with any users • Study existing knowledge and data • Ask users to gather their own data • Diary studies • Gather data through observing users • Ethnography • Contextual inquiries • Interact with User • Interviews • Focus groups

8. Diary Study

12. Advantages of Diary Study • Gather lots of data • Data gathers itself! • Gather data from many people at once • Easy to gather data over multiple sessions, or over time • Can worry less about experimenter bias • Experimenter isn’t present during data collection • Many users love doing it

13. Disadvantages of Diary Study • Users can forget to record their data, or get bored of the study • Think about how you will motivate them • Think about how much effort they need to participate, only focus on most important • Users may lose recording materials or data • Consider when choosing equipment • Have check points where user gives data • You may gather too much data • Make an analysis plan early

14. Ethnography

15. Ethnography • Primary technique from cultural anthropology Traditional science focuses on objectivity • Understanding from the “outside” • Ethnography focuses on understanding the subject as s/he understands her/himself

16. Ethnography • Studying actual habits and practices of stakeholders in context • Leave the lab and go find out what is really happening! • Shift from learning about behaviors from lab studies to real world experiences • This can produce richer and more realistic data about users • Ethnographer must create detailed records of observations • Notes, discrete video or audio recording • Ethnographers must take an unbaised and open-ended view of situation. • Ethnographers are observers, not actors • Roots in Anthropology and Sociology

18. Advantages of Ethnography • Your experience is your data • Can give researcher emotional empathy • Can provide huge amount of context • Experiences can be translated literally to design, or feed scenarios

19. Disadvantages of Ethnography • Very time consuming • Your experience is your only data • Heavily reliant on reflection • Individual experience, difficult for group participation

21. Extreme Ethnography: Patricia Moore • At age 26, she transformed herself into a range of women over the age of 80. The disguises involved more than makeup and clothing: She altered her body with prosthetics that blurred her vision, reduced her ability to hear and limited her motion. She relied on canes, walkers and a wheelchair. • From 1979 to 1982, she was in the roles about every third day for as much as 20 hours at a time. The experiment took her to 116 cities in 14 states and two Canadian provinces. • Used experience in communication design, product development, environmental design, package design, transportation design, market analysis and product positioning. • Video: http://www.youtube.com/watch?v=B4eOyBki3cE • Debatable if this is technically an ethnography…

22. Contextual Inquiry Hugh Beyer Karen Holtzblatt

23. What is Contextual Inquiry? • “The core premise of Contextual Inquiry is very simple: go where the customer works, observe the customer as he or she works, and talk to the customer about the work.” —Contextual Design (p. 41)

24. Contextual Inquiry vs. Ethnography • “...the intention is to understand and to interpret the data gathered, and rather than attempting to take an open-ended view, the investigator acknowledges and challenges her particular focus.” —HCI (p. 471)

25. What is CI? • Purpose: obtain data from the users in their context • Insights about users’ environment • Insights about users’ many tasks • Insights about the people they work with and work groups • Insights into cultural influences on work (expectations, norms, policies, values, etc.) • Goal: To help define requirements, validate design ideas, and prioritize support

26. Contextual Inquiry’s Roots • Evolved from ethnography • Observe from the “inside”, as a member of the community • Design to the needs of the user for current practice • Fundamentally an interview, but key is you interview in context

27. Objective vs. Contextual Questions • Objective questions: get at facts (what) • Contextual questions: get at how and why

28. Example Contextual Questions • What is important to the user? • How, why are certain choices made? • How does the user think about their needs and goals? • How does the user understand the system? • Note: user may not always understand her/himself, so we study the understanding, and study her/him objectively (in context).

29. Contextual Inquiry Stages

30. CI Stages: #1 Interview Warm Up • Introduce yourself, describe interests and the CI approach • Promise confidentiality • Get permission to tape • Start recording

31. CI Stages: #2 Transition • Re-state rules for the CI • Get the customer to do their work • Set ground rules (e.g., you ask questions, they can defer)

32. CI Stages: #3 Do the CI • Observe • Record artifacts, usage, temporal sequences • Create, share, validate interpretation • Refine interpretation (based on validation) • Follow up and ask questions

33. Writhdrawal and Return • Try to follow a pattern that Contextual Design calls withdrawal and return • •Researcher observes a key moment in the pattern of action • •Researcher asks about this, and the pair withdraw momentarily from the task • •Pair discuss issue (refine interpretations) •Afterwards, participant returns to the task

34. CI Stages: #4 Wrap-up • Skim back through your notes; summarize what you have learned • Validate your ideas • Understand customer’s fit within the organization

35. Principles of Contextual Inquiry

36. Principles of Contextual Inquiry • 1. Context: Understand the user’s work in a situated, real-world environment • 2. Partnership: Develop rapport and a joint approach to design w/ user • 3. Interpretation: Develop meaning and validate your observations w/ user • 4. Focus: Develop a topic range and maintain a clear focus

37. Things to Avoid When Conducting a CI!

38. Don’t be Afraid of Being Too Nosy • Don’t be too polite! • If you are curious, ask right away. • If you ask later, it might be too late; the participant may already have forgotten about it. • If you don’t know, don’t guess.

39. Don’t Disrupt the Task • Keep in mind the task is work. • Good times not to ask questions: • Phone conversations • Operating machinery • Remembering data or process • Think about asking people to take regularly scheduled discussion breaks.

40. Retrospective Contextual Inquiry • Retrospective methods can be a nice if you can’t interrupt, process is intermittent, or you can’t observe it • Tape / record task and have them view it • Focus on the past; not future • User brings relevant documents and objects • Step through process • Repeatedly query them about what happened in between the steps they recall / viewed

41. Respect the Participant • Be sensitive to cues from participant. • Remember: they are the expert at what they do. • Recognize and respect the time and effort they are spending to participate. • Respect boundaries participant sets.

42. Interview the right person • You want to interview the person who is directly involved in the task you want to study. • It is also helpful to interview someone who is positive about the task at hand, and about participating in a CI.

43. Interviewing Users Buy this book! Slides adapted from Lisa Anthony, Ravi Kuber, Lauren Serota, Amy Hurst

44. Avoid leading questions Avoid questions that suggest or “leads” the user to the answer or contains the information the examiner is looking for. Spot the leading questions "Do you agree that we need to save the whales?” “What do you think about the state of the whales?” “How would this software improve your commute?” “Would this software impact your commute?” “What impressions did you have about the interface?” “What was your favorite feature in the interface?”

45. Avoid leading questions Think of a question you are going to ask your participants about your interface Try writing it in two different ways: • Ask the question in a neutral way • Prime the user to give a positive (or negative) answer?

46. How to avoid leading questions Try and stay away from “yes/no” questions Instead, ask questions that encourage users to use their own words and think Be careful not to give away your opinions, or study goals in questions. Instead, stay neutral. Ideally, the participant shouldn’t realize you had anything to do with the design of the system.

47. Focus Groups

48. Why focus groups • Create a “design team” • Empower participants to engage with creative ideas • Find commonalities, trigger memories of focus group members • “Oh yeah, I’ve had that experience too…” • Can be more time efficient than interviews (but usually isn’t)

49. Why focus groups are hard • One person dominates conversation • Shy participants hesitant to engage • Run out of things to talk about

50. Why focus groups are hard • One person dominates conversation • Shy participants hesitant to engage • Run out of things to talk about • Turn taking, sub-teams, ask focused questions to everyone • Structured meetings (“today we’ll talk about riding the bus”)