Gathering User Data

1. Gathering User Data IS 588 Dr. Dania Bilal Spring 2008

2. Getting to know users • Setting goals for data collection • Working with research participants • Consent forms (IRBs) (signed by adults) • Assent forms (signed by children) • Choosing from various methods • Quantitative and qualitative

3. Methods • Interviews • Focus groups • Observation • Surveys and survey questionnaires • Field studies (in Evaluation) • Transaction logs • Experiments (in Evaluation)

4. Interviews • Structured interviews • Predetermined closed questions established • Unstructured interviews • Open questions • Semi-structured interviews • Open and closed questions

5. Planning an interview • Develop a set of questions based on goals and objectives of data gathering • Open vs. closed questions • Combination of open and closed questions • Questions to be written according to user’s level of understanding • Explanations provided as needed

6. Types of interviews • Face-to-face individual interviews with answers handwritten on interview question form • Face-to-face and audio-recorded interviews • Telephone interview with or without audio-recording • Diaries as basis for interviews • Qualitative method

7. Focus groups interview • Qualitative method • Focus group interview • 3-10 people • A group (or different groups) with a focus • May be from diverse user groups representative of target population • Requires a well-trained facilitator • Gather opinions, not behavior • Based on preset questions and agenda • Facilitator can follow up on unanticipated issues, ideas, etc. • Interviews are recorded, transcribed • Participants can be invited to listen to recording or read transcribed interview to explain comments or clarify certain responses • Can be expensive and time consuming

8. Interview questions • Exploratory: Open questions • Typically unstructured questions • Possible answers: closed questions • Typically structured questions • Combination of both types

9. Guidelines for interview questions • Ask clear, well-worded, non-leading questions • Do not ask double barrel questions • Ask questions that encourage participants to state opinion, express feelings, as applicable • Pilot-test, refine, and use questions

10. Running the interview • Introduce yourself • Introduce purpose of interview • Participant to read and sign consent form • Mention time interview may take in consent form and verbally • Participant’s signature confirms willingness to participate • Ask easy questions first such as demographic ones (warm-up session) • Opening questions • Main questions • Closed questions • Thank you note

11. Observation • Quantitative technique • Gather information about behavior • Provide information about users’ context, tasks, goals, success, difficulties, etc. • Two types: • Direct • Indirect

12. Direct observation • In a user’s field (natural setting) • In a controlled environment (setting chosen by observer: lab, café, etc. ) • In the field: Provides detail that can’t be captured through qualitative method • Requires good planning: structure and focus • Think of who you are observing • What you are observing • Where the observation is taking place (get to know the environment) • See text, p. 325 for detailed framework.

13. Direct observation: Controlled environment • Typically takes place in a usability lab. • Can be performed in a setting using a portable usability lab and/or software to capture user actions, verbalization, and feelings • Detail will be covered in Evaluation

14. Indirect observation • Tracking users’ activities • Transaction logs • Diaries/journals • Advantages/disadvantages of interaction logs • Advantages/disadvantages of diaries/journals

15. Choosing and combining methods • Variables to consider • Goal and focus of data gathering • Participants involved (who they are, how many, where they are, accessibility) • Time for data collection, transcription, coding, and analysis • Resources available • Other?

16. Survey and questionnaires • Technique for collecting data and users’ opinions • Used to capture data from group spread across a wide or small geographic area or place • Level of success based on type and nature of questions asked

17. Questionnaires • Questions can be like those used in structured or semi- structured interviews • Closed questions • Open-ended questions • Combination of both types of questions

18. Questionnaire design • Introduction • Instructions on how to complete questionnaire • Question sequencing • Length of questionnaire • Check boxes • Scales (Likert; semantic differential scale) • Yes, No, Don’t Know