Qualitative and Quantitative Methods of Survey Design What to do from start to finish
ATopic • The first step is to identify a specific research question and then become an expert in providing the answer. • I keep a working list of everything that I want to know---every single question. • Example: Are there differences among men and women? Old and young? By Social class.
How to Go About Being an Expert • A point of agreement: a researcher must be fully informed about a phenomenon in order to measure it in a way that produces rich quantitative data ! • I have found it extremely critical to learn how people experience, think, and talk about a topic BEFORE formulating (quantitative) questionnaire items to answer all of my research questions.
The most productive qualitative methods: Personal interviews—one on one. • Without an interview schedule. • With an interview schedule. • Group interviews/focus groups • With an interview schedule
How to use the interview • Identify the population of interest. It helps to identify or find typical cases. • Begin without a list of questions. • Take the subject(s) on a mental tour. • A mental tour involves placing subjects in their setting, and recalling the time and place where they experience the phenomenon one wishes to study.
Qualitative Methods • Yes, one should tape record OR have someone manually record a subject’s words. • Yes, one must have Human Subject’s approval • The task is to listen and probe: • Example: Can you think of other instances when _________. How is this different than _____________. • Comparison questions are ALWAYS helpful.
Qualitative Methods • After the interview, I read the transcript, and ask myself these types of questions: Who else experiences this in the same way? Who might experience it differently? What else is involved in this experience? What is the context? What is the role of context?
James Spradley’s Matrix • Space • Object • Act • Activity • Event • Time • Actor • Goal • Feeling
Qualitative Methods • The ideal number of interviews is…. • The ideal is to continue doing these until one no longer learns new information.
Qualitative Methods • Focus groups are more efficient, but more difficult to organize. • Focus groups are small (6-10), structured, group interviews where subjects talk to each other rather than to the researcher. • Focus groups benefit from a trained moderator.
How many focus groups? • As many as one can afford, OR • One continue until no new information emerges.
An aside: • Personal interviewees and focus group members identify people with a potential to become the researcher’s consultants! • They can pretest surveys • They can assist in providing qualitative interpretations on your quantitative data
What next? • Qualitative data provides the many dimensions---and the ways of talking about all aspects of a research question. • I use the information gleaned from a close reading of the transcripts to reformulate my research questions and to formulate my questionnaire items.
Five kinds of measures • What kind of information* might I measure? • What people say that they do (behaviors) • What people think is true (beliefs) • What people are (attributes) • What people say they want (their attitudes) * Dillman, Don A. (1978). Mail and Telephone Survyes: The Total Design Method. NYC: John Wiley. P. 80
Examples: • Behaviors : • On an average weekday, how much times do you think about your breast cancer? • Attitudes • In your opinion, how do you feel about the legalization of marijuana? • Strongly oppose/Somewhat oppose/neither oppose or favor/favor somewhat/strongly favor • Beliefs • In your opinion, does chemotherapy always cause hair loss? • Always Yes/Usually Yes/Sometimes/Seldom/Almost Never/Never
Types of Qs, cont. • Attributes/demographic/Profile Questions: • What is your marital status? • 1 Never married 2 Married • 3 Separated 4 Divorced • 5 Widowed • How many years of education have you completed? Please tell us the actual number of years: • __ __ years
Content: Types of Q’s • Open-ended questions: subject write their answers in a few words • Describe some of the things that you do around the house_________________ • Closed-ended questions: coded categories are provided—these can be analyzed more easily • Please circle all of the things the things that you have done in your household during the past seven days: • 1 Bought groceries 2 Packed lunches • 3 Cleaned 4 Ironed • 5 Cooked 5 Made bed
Open-ended Questions • Should be used sparingly. • Are more appropriate for an exploratory study. • Data must be encoded so that it can be analyzed • This is time-consuming • Reliable codes is an issue
Closed-ended Questions • The responses to closed-ended questions can be entered directly into a statistical program or spread sheet. • Closed-ended questions require categories that are mutually exclusive (only one response category per question) and exhaustive (a category for each person).
Content: Writing the Qs • Language: Most people agree that questionnaires should be written in the language of the population of interest. • If qualitative data are the basis of the questionnaire items, this should be relatively straightforward. • When it comes to forming items for the questionnaire, here are a few things I check, recheck, and check again: • That language is neutral, not emotional • That the subject is not led to socially desirable answers. • That the language is unbiased • That words are short---not too many letters or syllables • That words are precise and specific rather than vague so that everyone understands the items the way one intends it. • That language is common rather than haughty or academic
What should I ask? • How to decide what to ask: • Keep a focus on the research question. • Keep that list of the related questions. • Make dummy tables!!!!!
Format • Questionnaires should be formatted to promote visual appeal. • PAY someone well to do this!!!!! • Consider the ‘flow’ of items • Use transitions or Headings • Include “codes” that can be entered directly into your statistical software. This increases the reliability of data entry.
Closing Thoughts • Research is creative---that’s the allure. • Everyone has to find their own style • Everyone has to find what works for them. • What I have shared today is what works well for me. Research is time-consuming---that’s the greatest difficulty. • Collaborate to reduce the amount of labor and to increase the fun. • Ask for help: Teri Peterson, Statistical Consultant; Ann Hunter, Sociology.