Conducting a survey

1. Conducting a survey Adapted from Del, Balso Michael, and Aland Lewis D. First Steps: A Guide to Social Research. Toronto: Nelson Thomson Learning. 2001. Print.

2. the aim and purpose of surveys • how to formulate effective questions to guide your research and inquiry • How to administer a survey, including how to analyse and interpret your results Learning goals:You will learn…

3. You can conduct a casual inquiry, whereby you ask people you know for the information you need. For example, if you wanted to know whether or not high school students with jobs did better or worse than those without jobs, you could ask some of the people in your class, or the ones you sit with at lunch, or others at football practice after school. • NUMEROUS FLAWS! • Social scientists would raise many questions about the research: • Which group were you interested in studying? • How did you choose the students to be included? • How did you determine academic performance? • How many courses do the students take? At what level? • How many hours did the students work in a week, on average? • Were the jobs in any way related to their studies, or were the employers flexible? • Is the information collected sufficient to make generalizations about all of the students in the school? • NO A simple method…

4. The scientific inquiry • Terms must be defined (e.g. what is meant by “working”, or “academic success”?) • The Aim and Purpose of Surveys • To describe the characteristics of a population • To study attitudes and opinions • To examine the relationship between two variables (e.g. the relationship between age and voting behaviour) • To test theories • Most surveys are ‘cross-sectional surveys’ – these are designed to find information about a population which is particular to a time and place (e.g. the Census) • If a survey is meant to track or predict changes, then a ‘longitudinal study’ is done • There are two kinds of longitudinal study: • The trend study – this is a survey done on a similar population each year (e.g. each year’s graduating class) • The panel study – this would revisit the same group of people at regular intervals to track changes in their attitudes of beliefs (e.g. students who entered O.P. in 2010 are surveyed; these students are re-questioned each September) So how to do a survey then?

5. Establish the main purpose of the survey • Determine your population • Select your sample from your population • Determine the technique for administering the questionnaire • Develop the questionnaire • Pretest the questionnaire, if possible. • Revise the questionnaire, if necessary. • Administer the questionnaire to the selected sample. • Organize, summarize, and analyze the collected data. How to conduct a survey

6. There are two basic elements that make a good questionnaire: • It should be clear to the respondents • It should measure what it is intended to measure • To achieve these goals, the questionnaire must provide operational definitions of terms – these go beyond the dictionary definitions so that they ask very precisely what is intended to be asked. • E.g. “Do you have a job?” • This ignores many of the considerations about what constitutes a job – is this meant to determine who works full time? Part time? Self-employment? Does a regular expectation to help the neighbour by mowing her lawn for $10 per week constitute a ‘job’? If your uncle owns a seasonal company for which you WILL work, but it is out of season, does that count? • “Do you presently work for an employer or business, for pay or profit?” • This would more likely yield useful results, as it more precisely defines what a ‘job’ would be. The questionnaire

7. Aim for clarity. Each questions must be phrased in language appropriate for the sampled population. • Do not ask questions that could reasonably be considered to be beyond their realm of knowledge. (e.g. ‘At what age did you learn the alphabet?’ or ‘Is the current Syrian regime consistent with other Mideastern regimes of the past 100 years?’ • Use both open-ended and closed-ended questions appropriately. • Open-ended questions allow the respondent to state a response in their own words; closed-ended questions have the respondents select from a list of choices. • Remember that the choices in a closed-ended questions must be exhaustive (they include all likely answers), they must be mutually exclusive (there is no overlap in choices) • Ensure that there is no ambiguity with your questions or responses • Avoid double-barrelled questions – these would force the respondent to answer two or more questions with a single answer. E.g. ‘Does Orchard Park have a pool and a theatre?’ The questionnaire

8. Avoid leading or biased questions. These can arise in a number of ways: • The question may encourage the respondent to give a particular answer. • The question may emphasize issues in a way that influences the answer. • The choices of responses may emphasize one response over another • The sequence of questions may direct the respondent to an answer. • Women should have the right to terminate an unwanted pregnancy. • Agree • Disagree • Strongly disagree • Very strongly disagree • Did you know that the university will have to increase student fees if it sets up a new computer lab? • Yes • No • Should the university set up a new computer lab for social science students? • Yes • No The questionnaire

9. Ask questions that are relevant to most respondents. • E.g. ‘Should the federal government restrict the number of international banks in Canada?’ • People will still answer the question, even though they have no legitimate understanding of what exactly is being asked. • Avoid negative items • E.g. ‘Do you agree or disagree that Quebec should not separate from Canada?’ The Questionnaire

10. Include introductory remarks • People taking a survey should know some background information about what the survey is meant to test, whether or not the responses are confidential, how long it should take to respond to the questions. • Provide instructions to the respondents • If someone does not know or does not have an opinion with which to respond to a question, should it be left blank? • Should answers be circled? Marked with an X? Can ALL applicable answers be chosen? • Be sure to format the questionnaire in an easy-to-read manner • If necessary, provide instructions on contingency questions • E.g. on a questionnaire that asks if the respondent smokes cigarretes, follow-up questions are likely not applicable to people who answer ‘no’ – be clear about how to proceed. • Pretest the questionnaire – ask a few friends or other students to take the test for the purpose of testing its effectiveness. Get feedback, and adjust as needed. Constructing the questionnaire