Focus Groups

1. Ciaran B. Trace MAC 2010 Symposium: Archival User Studies Dayton, Ohio, October 21-23, 2010 Focus Groups

2. Outline of the presentation • Introduction to focus groups • Before you begin • Carrying out the focus group • Data analysis • Sum up • Questions?

3. Resources used in this presentation • Richard A. Krueger and Mary Anne Casey, Focus Groups: A Practical Guide for Applied Research, Sage: Los Angeles, 2008. • David W. Stewart, Prem N. Shamdasani, Dennis W. Rook, Focus Groups: Theory and Practice, Sage: Thousand Oaks, 2007. • Denise Troll Covey, Usage and Usability Assessment: Library Practices and  Concerns Washington, D.C.: Digital Library Federation Council on Library and Information Resources, January 2002. Available at http://www.clir.org/pubs/reports/pub105/pub105.pdf) • Daniel L. Morgan, Focus Groups as Qualitative Research, Sage: Thousand Oaks, 1997.

4. What are focus groups? • “A focus group is an exploratory, guided interview or interactive conversation among seven to ten participants with common interests or characteristics. The purpose of a focus group is to test hypotheses; reveal what beliefs the group holds about a particular product, service, or opportunity and why; or to uncover detailed information about complex issues or behaviors from the group’s perspective. Focus group studies entail several such group conversations to identify trends and patterns in perception across groups. Careful analysis of the discussions reveals insights into how each group perceives the topic of discussion.” [Covey 2002]

5. Key elements • Notion of an interactive conversation among a small number of participants • Not focused on the experiences and opinions of any one participant [would need to do interviews for this] • Evidence of the differences and similarities in the group • More than one focus group held in order to properly explore trends and patterns in perception across groups (each focus group will last anywhere between one and two hours) • Need participants and also a moderator and an observer • Need to record data and subsequently analyze that data

6. What can focus groups be used for? • Determining patron satisfaction with resources and services • Helping to shape the future direction of an archive as part of the strategic planning process • Developing and evaluating web and catalog interfaces • Developing and evaluating user education or reference services

7. Things to note… • A form of qualitative research method (with all that that entails) • Moderator has a distinctive role • Interaction occurs in a group setting • There isn’t a very high degree of naturalism • Generally, you are recording not what people do, prefer etc….. But what they say they do, say they prefer, etc… Data is verbal and self-reported • Small number of participants, selection criteria, and frequent use of convenience sampling mean that you cannot generalize the results • The results are generally intelligible and accessible to all audiences

8. Things to note… • Can be used as a self-contained method, as an initial/ supplementary source of data, or as part of a multi-method approach (paired with surveys, individual interviews, participant observation etc) • Paired with surveys – focus groups as a means to construct questionnaires (domains, dimensions, wording) • Paired with individual interviews – focus groups as a means for devising the interview schedule for unfamiliar topics and vice versa • Paired with participant observation – focus groups as a tool to help you chose among potential field sites • As you can see…. focus groups can be used as a means to initially identify appropriate language, questions, or tasks, and as a follow-up tool to get clarification

9. Before you begin… • Articulate the purpose of the focus group • Why is there a need for a study such as this? • What is the problem that the study seeks to address? • What kinds of information do you want to get from the study? • What wants the information? • How will the information be used?

10. Before you begin… • Figure out what types of people can give you the information that you want • Who has the greatest amount of insight on this issue? • What particular characteristics should these people have?

11. Before you begin… • Figure you how many focus groups you need to hold and how they should be sequenced • Usually hold 3 or 4 focus groups – allows you to be sure that you have reached saturation point and allows for data to be compared across groups • If you want to compare and contrast how certain types of people talk about an issue separate them into different groups • If you anticipate that participants will have conflicting opinions ,hold focus groups in two stages (separate the groups initially and then bring them together)

12. Before you begin… • Determine how you will capture the focus group data • At minimum, will have an observer taking notes • Can also audio tape the focus groups (transcribe partially or in toto?) • Can also video tape the focus groups (transcribe partially or in toto?) • Video taping is rarely done – it is considered too intrusive • Can take a combined approach – notes and audio, for example

13. Before you begin… • Think about the ethical issues and determine if you need to obtain human subjects clearance… • Nature of the group setting brings up privacy concerns – limits the kind of topics the researcher can pursue • Will need to get explicit permission from participants to tape (audio and video) • Decide up front who will be able to listen to the audio tapes (should limit access to research staff) • Make decisions about how the data will be used so that this information can be conveyed to the participants

14. Recruit and schedule participants • Using the criteria you have already identified, recruit and schedule participants. • Not looking for generalizability but to minimize bias • Not random sampling but theoretically motivated sampling or segmentation • Remember that focus groups are composed of people who are similar to each other in a way that matters to the researcher – homogeneity is important but you need enough diversity to allow for contrasting opinions • Involves scheduling six to twelve participants in designated demographic groups • Usually avoid mixing people with different levels of power or expertise • “Ideally” participants should not know each other • May want to “pre-screen” to make sure the participants have the requisite characteristics

15. Prepare interview questions • Prepare interview questions…. • Think about the types of questions you need to ask ….and how you are going to sequence the questions • Also think about what makes for good questions….. • Normally ask about a dozen questions for a two hour focus group • Remember that the interview guide will have a direct bearing on the content of the group discussion

16. Good questions… • Good questions are… • Open ended and one dimensional • Sequenced (e.g. general before specific, positive before negative etc) • Evoke conversation and allow for depth (ask participants to think back, avoid asking them “why”) • Engage participants (can go beyond oral questions and engage people in doing things – listing, rating, choosing among alternatives)

17. Good questions… • Good questions… • Use words that participants would use when talking about the issue, are easy to say and easy to understand, are short, are clear, include well thought out directions…..

18. Types of questions used in focus groups • Types of questions used in focus groups • 1. Opening question – icebreaker, for all participants, question should be designed to be easily and quickly answered (30 seconds or less), shouldn’t be used to highlight differences among the group (age, occupation, educational level), generally not included in the data analysis

19. Types of questions used in focus groups • Types of questions used in focus groups • 2. Introductory questions – open ended questions that introduce the topic of discussion, get people thinking about their connection to the topic • Should be questions that interest all participants • Method - ask people to remember when they first encountered the organization, service etc and to describe the experience. Or “what is the first thing that comes to mind when you hear the phrase….”

20. Types of questions used in focus groups • Types of questions used in focus groups • 3. Key questions - focus on the key areas of concerns. Most of the questions fall into this category. • 4. Ending question(s) - E.g. is there anything we should have talked about but didn’t.

21. Role of moderator • Recruit, schedule, and train a moderator and observer(s) for each focus group • Skilled moderator is key - develop rapport with the participant, remain impartial, and keep the discussion moving and focused on the research objectives • Moderator must have knowledge of the background topic • Moderators must be able to handle the human element… • Moderators and observers must avoid doing anything that may bias the results of the study • Goal is to encourage conversation among the participants – NOT between the moderator and the participants

22. Role of observer • Main task is to observe the session and to take extensive notes • Not only capturing what is said but also documenting nonverbal signs and body language – tells you a lot about how the group feels about the topic as well as giving some indication of how many people hold the same idea • Notes should therefore include meta-comments, e.g. “Jim seemed surprised at Angela’s response…”

23. Handling logistical issues • Handle all logistical issues associated with running focus groups • Securing compensation?, scheduling rooms, ordering food/drink, having copies of focus group questions for moderator and observer, testing the audio- or videotape equipment, creating name placards for all those taking part in the focus group • On the day - getting the room ready (light and temperature), setting up and testing equipment, bringing supplies (handouts, name placards etc)

24. On the day…introduction, welcome, background • The actual focus group itself…role of the moderator • Welcomes the participants and thanks them for participating, introduces and explains their role as moderator and the role of the observer, reiterates the purpose of the focus group, confirms human subject requirements (anonymity, taping, how data will be used etc), briefly describes the ground rules and how the focus group will be conducted, emphasizes that the goal of the focus group is not to reach consensus - disagreements and discussion are important

25. NCSU sample focus group script (introduction, welcome, background) North Carolina State University Library http://search.ebscohost.com.www.lib.ncsu.edu/userstudies/documents/ • Thanks for taking time to join us to talk about some library tools, services, and resources that can help students with their class assignments. We here at the library are hoping to develop a suite of web tools to assist students with their learning experience. We really want to make these web pages as useful for students as possible. That's why we've asked you to come talk with us. You've probably noticed our microphone. We're going to tape record this session because we don't want to miss any of your comments. People often say helpful things in these sessions that we can't write down fast enough plus we don't want you to feel that you need to talk at a pace we can write. Please feel free to talk casually and naturally. (Co-moderator) and I will take what we learn from your tonight and share your ideas with our colleagues working on these web pages. Again, this is meant to be a relaxed discussion so we will be on a first name basis for the duration of this session. Each of you as you can see has a name placard in front of you. Hopefully the names on them are correct. We assure you that your names will not be linked in any way to any written materials produced from this focus group.

26. NCSU sample focus group script (introduction, welcome, background) • We're going to ask you look at some content and tools that we are thinking about including on these web pages. First we will provide a static version and would like you to rank them by your first impression of their function, and we will discuss as a group which of the pieces of content seem most interesting and helpful from your perspective and how you might envision yourself using each piece. You can even suggest new pieces of content that we might have not have thought of. After this initial exercise and discussion we will have each of you revisit each piece of content and explore their functionality as currently designed using the laptops in front of you. Try and note mentally how using these tools reinforces or dilutes the power of the tool you had initially assessed. We would like you to once again rank your top fives content pieces, singling out one as the most valuable. We will then discuss each other's rankings and how things may have changed. We feel that these activities and discussions will help us dig into what you think about tools, services, and resources that are helpful to you or that you can imagine being helpful to other students. There are no wrong answers for what we're going to discuss. We know you might have differing points of view and we value everyone's ideas so we really want to hear them.

27. First and subsequent questions…. • Sometimes the first question is asked round-robin and is a question that each participant can answer. In this way each participant responds and gets comfortable talking. • Subsequent questions are answered less formally, and more conversationally. • Moderator clarifies anything that participants do not understand. • The moderator may invite further comments to better understand what the participants are saying, For example, “Would you explain that further?” or “Please give me an example.” • The moderator uses verbal and body language to invite comments from shy participants and to discourage others who may be trying to dominate the conversation.

28. Subsequent questions…. • The probe should also be paired with another important tool…. the pause. Pause used as a way of eliciting a response from the participants. • Moderator must be comfortable with these silences.

29. Data analysis

30. Data analysis and reporting • You might or might not transcribe the focus group tapes - cost of transcribing may exceed the benefits of having a full transcription • At minimum, can use the tapes to annotate the observer’s notes • Ideally, at least two people analyze the data—the moderator and observer—high inter-rater reliability • Goal is to examine, categorize and make sense of the data.

31. Data analysis and reporting • Review – doing a thorough read through of the data – notes, transcripts, etc • Conduct content analysis on the data • Initial coding – involves reading the data and assigning labels or codes to each piece of relevant information • Can be done by hand, using note cards, or using a computer. • Choice of strategies - note all mentions of the code, note whether each individual mentioned a given code, note whether each groups discussion mentioned a particular code • Reduce this information to manageable aggregates by creating memos for key themes that emerge from this initial coding • Test the themes by going back to look at the data

32. Data analysis and reporting • Reporting…. • Which topics to emphasize in the report • How many groups mentioned the topic • How many people within each group mentioned the topic • Degree of enthusiasm displayed for the topic • Notion of “group-to-group” validation • Blend of summary of discussions and direct reporting of comments

33. To sum up… Advantages of focus groups Advantages of focus groups Group discussion creates synergy and facilitates recall and insight Direct interaction with participants allows for clarification of responses, probing of responses, and follow-up questions Results of focus group research are user friendly and easy to understand • A way to gather insight into behavior and experience from the participants’ perspective • Allows for the gathering of rich data, in the participants own words • Produce a concentrated amount of data on a precise topic quickly and at relatively low cost

34. To sum up… Disadvantages of focus groups Disadvantages of focus groups Quality of the information is impacted if participants are unduly influenced by others, stray off topic, or seek consensus Hawthorne effect - cannot observe something without changing it Findings have to be interpreted at the group level and are not generalizable • Quality of the responses depends on clarity of the questions, the moderator’s skills, and the participants’ understanding of the goals of the study and what is expected of them • Rely heavily on the observational skills of the moderator and observer

35. Questions…