Survey Designs

1. Survey Designs Chapter 12 Lauri Cabral Sean Lafontaine

2. Outline – SURVEY RESEARCH • What is it? • What types are there? • What are the features? • Types / Features / Design and Construction • What are the ethical considerations? • What are the steps involved? • How do you evaluate and analyze them? • What are their strengths & limitations?

3. What is Survey Research? • A form of quantitative research in which researchers administer a survey to a sample or to the entire population to describe attitudes, opinions, behaviours, or characteristics of the population • Collecting quantitative data using questionnaires, researchers statistically analyze the data to describe trends about responses to questions and to test research questions and/or hypotheses • Similar to correlational designs because surveys do not explain cause and effect as experimental research does

4. Translation....

5. Surveys • Surveys are effective and efficient instruments used by researchers to gather information on trends, attitudes, and characteristics of a population

6. When Do you use Survey Research? • To describe trends • E.g. trends about levels of bullying in schools • To identify individual opinions about policy issues • E.g. Your opinion on Bill 115 • To evaluate the level of success of programs • E.g. The success of the breakfast program • To identify important beliefs and attitudes of a population • E.g. student beliefs about what constitutes bullying

7. 2 types of Survey Designs Longitudinal • Trend Studies: examine change within a population over time • E.g. Level of bullying in certain schools • Cohort Studies: a sub-group of a population that has a common defining characteristic • E.g. Parents level of education • Panel Studies: the same people are examined over time • E.g. The 7 Up Project Cross Sectional • Collects data at one point in time • Measures: • currentattitudes, beliefs, opinions, or behaviours of one or more groups OR • community needs OR • evaluate programs

8. Types of cross-sectional and longitudinal survey designs (figure 12.1) Time of Data Collection Longitudinal Cross-Section Community Needs Program Evaluation National Assessment Trends Cohort Panel Attitudes and Practices Group Comparisons

9. Key characteristics of survey research • Sampling from a population • Collecting data through questionnaire or interviews • Designing instruments for data collection • Obtaining a high response rate

10. SAMPLING FROM A POPULATION

11. To maximize survey results • Ensure a good target population • Select a large enough sample • Have clear, unambiguous questions and response options • Use rigorous administrative procedures to achieve a large return rate. • RANDOM SAMPLING is the most rigorous form of sampling .... Exception: census studies

12. Collecting dataQuestionnaires and interviews • Interview Survey A form on which the researcher records answers supplied by the participant. • Questions are asked from an interview guide • Researcher listens for answers or observes behaviours and records responses • The investigator uses a structured or semi-structured interview consisting mostly of closed-ended questions, provides response options to interviewees and records their responses.

13. 2. Types of questionnaire surveys • Mailed Questionnaires: • Web based questionnaires: • One on one interviews: • Focus group interviews • Telephone interviews Youth Bullying Survey

14. 3. Instrument design Options: • Consider using an existing survey instrument • Consider modifying an existing tool • When neither 1 nor 2 are options, design your own instrument

15. Designing an instrument for data collection • Write different types of questions • Personal, Attitudinal and Behavioural questions • Sensitive questions • Use strategies for good question construction • Closed questions • Open-ended questions • Semi-structured questions • Perform a pilot test of the questions

16. Question construction

17. Question construction • Must be clear and unambiguous • Sensitive to gender, class and cultural needs of participants • Ensure question has a clear response and that the participant will be able to answer

18. common construction problems • Question is unclear • Multiple questions within a single question • Question is too wordy • Question is negatively worded • Question includes jargon • There are overlapping responses • Unbalanced response options • A question includes overly technical language • Not all questions are applicable to all participants

19. Pilot testing • A test run of your survey to ensure it does what it was designed to do • Always pilot test your survey! • E.g. Neighbourhood Aesthetics Observational Survey

20. Response rate • Researchers seek a high response rate that is not biased • A high response rate creates a stronger claim in generalizing results • Interviews obtain higher response rates • To encourage high response in questionnaires, certain methods are used: • Pre-notify participants that they will receive a questionnaire • Use good follow-up procedures • Study a problem of interest to the population being studied • Use a brief instrument • Consider the use of incentives

21. Response bias • Occurs when the responses do not accurately reflect the views of the sample and the population • Wave analysis is a procedure to check for response bias. Investigators group returns by intervals and check to see if the answers to a few select questions change from the first week to the final week of the study • E.g. Failing a test in the middle of a month may change a student’s attitudes toward the teacher

22. Constructing a mailed questionnaire • The Cover letter: • Highlights importance of participant and value of response • Purpose of the study, including informed consent • Assurances of confidentiality • Sponsorship • Completion time and returns

23. Tips for questionnaire construction • Short length • Begins with straightforward demographic or personal questions that encourage commitment • Use a variety of closed ended questions • Some open-ended items to encourage elaboration • Pleasing layout • Closing instructions thanking the participant

24. Data analysis of a questionnaire ( Fig.12.7) • Identify response rate and response bias • Descriptively analyse the data to identify general trends • Write the report presenting the descriptive results and/or use advanced statistics

25. Potential ethical issues • Survey research may be exempt from a detailed review by review boards (unless it addresses sensitive topics or minor populations) • Incentives can be used, but should not be large • Do not overstate the benefits of participating • Do not put interviewee safety at risk or be deceptive, misleading or inaccurate. • Confidentiality of respondents must be maintained. • Links between answers and participants should be made with an ID number that is only known by the researcher. • Be careful when reporting a small subset of results that could disclose identity • When the project concludes, you are responsible for destruction of detailed/personal survey instrument information

26. Steps in survey research

27. Examples • Neighbourhood Aesthetics Observational Survey • Personality Test www.personalitytest.net/ipip/ipipneo1.htm • www.prevnet.ca/BullyingFacts/BullyingStatistics/tabid/122/Default.aspx

28. Thank You!

29. Discussion • Strengths? • Weaknesses? • What do you think? • Questions?