Surveys and Questionnaires

1. Surveys and Questionnaires See Robson Chapter 8

2. Typical Survey Features • Use of a fixed quantitative design • Collection of a small amount of data in a standard form from a large number • Selection of representative samples from known populations • Generally closed-ended • Not hard and fast rules

3. Survey Types • In-Person Questionnaire • Self-Administered Postal (mail) Questionnaire • Telephone Survey • Not restricted to the use of questionnaires • E.g. traffic survey – observational

4. Advantages of Surveys • All Survey Types: • Simple and straightforward approach • Adaptable to collect generalizeable information from almost any population • Data standardization • Interview Surveys: • Interviewer can clarify questions • Presence of interviewer encourages participation

5. Advantages Cont. • Self-Administered/Mail: • Way of retrieving information about history of a large group • Efficient – Large data sets, low cost, short timeframe • Note: May be disadvantage if researcher is seduced into using a survey when it is not most appropriate to research question • Allows anonymity/encourages frankness • Allows large geographical distribution

6. Disadvantages of Surveys • All Survey Types: • Context, discourse, and meaning • Lack of relation between attitude and behavior • Social desirability response bias • Data affected by characteristics of respondents • Self-Administered/Mail: • Low response rate – still representative? • Misunderstandings not detected • No control • Respondents may not take seriously

7. Disadvantages Cont. • Interview Surveys: • Data affected by interviewers/interview bias • Affected by interactions of interviewer/respondent characteristics • Concerns of anonymity – respondent less open

8. Developing Survey Questions • Pilot work – semi-structured interviews, focus groups, other data collection • Previous studies • Theoretical frameworks

9. Activities in a Sample Survey • Initial design and planning • Designing the questionnaire • Pre-Testing • Final design and planning • Data collection • Analysis and reporting

10. 1. Initial Design and Planning • Most straightforward task for survey are ‘how many,’ ‘how much,’ ‘who,’ ‘where,’ and ‘when’ • Gauge public opinion • Test theories • Theoretical framework important when trying to move beyond description to explanation • Unit of analysis • Population • Sampling frame

11. 2. Designing the Questionnaire • Provides a valid measure of the research questions • Gets the cooperation of respondents • Elicits accurate information • Be clear • Fixed-alternate responses should be accurate, exhaustive, mutually exclusive, and on a single dimension • Keep response rate in mind for mail surveys • Standardize in-person surveys

12. 3. Pre-Testing • First stage: • Informal – Colleagues, friends, family • Ask if questions are clear, simple, and unambiguous • Second stage: • Respondents from the groups of interest • Give thoughts on the question • Third Stage • Formal Pre-test • Pilot version

13. 4. Final Design and Planning • Edit questionnaire • Spelling • Layout • Finalize coding and analysis procedures

14. 5. Data Collection • Follow your plan! • Keep clear notes/records • Address issues as they arise

15. 6. Analysis • Generally straightforward with closed questions • Codes can be arbitrary (but consistent) or can be the actual number • Have a code for non-response • Open questions – Simplify many responses by classifying into smaller number of groups • Coding categories chosen from sample of all respondents • Coding effectively changes open questions to defined set of responses

16. Survey Sampling • Selection from the ‘population’ • Sample size • Would a smaller set of longer more detailed surveys be preferable to more? • Probability/representative samples • Non-Probability samples

17. Interviews See Robson Chapter 9

18. Interview Features • Generally one-on-one and face-to-face • Can be in group settings or by phone • Can be primary/only approach or in combination with other methods • Open-ended questions are the norm

19. Types of Interviews • Structured • Extreme example is the survey • Semi-Structured • Unstructured • Allows more flexibility of response • Extreme is the ‘depth interview’ • Respondent interviews/informant interviews

20. Advantages of Interviews • Flexible and adaptable • Asking people directly is a short-cut in seeking answers (versus observation) • Offers possibility of follow-up • Non-verbal cues • Rich and highly illuminating material

21. Disadvantages of Interviews • Requires considerable skill/experience • Lack of standardization/reliability concerns/bias • Time consuming and requires lots of preparation • Can be limiting geographically

22. Interview Process • Listen more than you speak • Questions should be straightforward, clear, non-threatening • Avoid leading • Enjoy it! • Take a full record • General format: Intro, Warm-up, Main Body, Cool-Off, Closure

23. Interview Content • Content consists of: • A set of items/questions • Closed • Open • Scale • Probes and prompts • Proposed sequence of questions

24. Other Interview Types • Informant interviews • Non-directive interview: Totally in control of interviewee • Focused interview: • Situational analysis • Interview guide • Group Interviews • Most common are a hybrid of discussion and interview • Allows group interaction • Focus Groups

25. Focus Group Advantages • Highly efficient • Checks and balances • Focus on most important topics • Enjoyable • Inexpensive, flexible, quick to set up • Stimulated by thoughts of others • Contributions encouraged • Those with limited reading/writing skills not excluded • Less inhibited members serve to break the ice

26. Focus Group Disadvantages • Limited number of questions • Facilitating requires considerable expertise • Needs to be well managed • Conflicts may arise • Lack of confidentiality • May not be generalizeable • Live and immediate • Participants may try to impress one another • Difficult to follow-up on views of individuals

27. Focus Groups • Homogeneous Groups: • Facilitates communication • Promotes exchange of ideas and experiences • Gives a sense of safety • May result in ‘groupthink’ • Heterogeneous Groups: • Can stimulate and enrich discussion • May inspire new ideas • Risk of power imbalances • Can lead to lack of respect for opinions • Can lead to dominant participant destroying group process

28. Analysis of Interview Data • Taping and Transcribing • Allows concentration on the interview • Alternative to complete transcription is to be selective • Too late to think about analysis when interviews are complete