Surveys

1. Surveys Introduction to Communication Research School of Communication Studies James Madison University Dr. Michael Smilowitz

2. Surveys • We are all familiar with surveys. • Political surveys (Gallup and Harris) • Commercial surveys (Pepsi challenge) • Media surveys (TV ratings

3. Advantages of Survey Research 1. Investigates problems in realistic settings. • Things like newspaper reading, supervisory-subordinate communication, and consumer behavior can be examined where they happen rather than in a laboratory. 2.Gathers data from a variety of people. • Not stuck with student populations. 3. Provides large amounts of data at reasonable cost. • Much of the work of mail surveys, or telephone surveys, or even face-to-face, can be done with cheap labor.

4. Disadvantages of Survey Research • For example, let’s say we want to test the hypothesis: • H1: Supervisors who use confirming communication with their subordinates encourage productivity. • If we survey, we might find that effective supervisors are regarded by their subordinates to be more confirming. But we don’t know if the confirming communication leads to the higher productivity. We only know they may be related. To see if confirming communication is responsible, we would have to control the levels of confirmation and then see the outcomes on the dependent measures 1. Does not provide control of independent-variable variation. • Without control, cannot tell if the relations between variables are causal or non-causal.

5. Disadvantages of Survey Research 1. Do not provide control of independent-variable variation. • Without control, cannot tell if the relations between variables are causal or non-causal. • 2.Inappropriate wording and placement of questions with a questionnaire can bias results. • Surveys are subject to “question contamination.” • 3. Can get data from the wrong people. • Don’t know for sure who fills out the survey unless you’re there. Particularly a problem with mail surveys, and phone surveys. • 4. Difficult to conduct. • People are survey battered - especially with phone surveys.

6. TYPES OF SURVEYS There are two pairs of distinctions to know: • Descriptive versus Analytical • Cross-sectional versus Longitudinal

7. TYPES OF SURVEYS • DescriptiveSurveys Provide information about the current conditions or attitudes of a group of people. • Examples: • A census surveys the population to determine demographic characteristics. • Political pollsters or commercial polls examine public attitudes about candidates, issues, products, media services, etc.

8. TYPES OF SURVEYS • Analytical Surveys Provide information that allows researchers to examine the interrelationships among variables to answer research questions.

9. TYPES OF SURVEYS • Analytical Surveys Provide information that allows researchers to examine the interrelationships among variables to answer research questions. For example, the ICA Audit of Organizational Communication allows tests of subordinate/ supervisory relations and perceptions of information adequacy. Media research often goes beyond describing viewers attitudes and preferences to examine the reasons for their opinions, by considering such things as psychographic variables, or how people’s lifestyles predict media choices.

10. TYPES OF SURVEYS • Cross-Sectional Surveys • Collects information from a sample at a given point in time. • Appropriate when there is no need to know about changes or trends. “One-shot surveys” is another name for cross sectional surveys.

11. TYPES OF SURVEYS • Longitudinal Surveys • Collect information at different points in time. • There are two types of longitudinal surveys. • Panel Studies which follow the same same respondents across time. • Trend Studies which select different represen-tative samples from the same population at different times.

12. Choosing between a panel or trend survey: 1. Size of the population. Trend studies are feasible only if the target population is quite large. If the sample population is small, do a panel study. 2. Nature of the research question. If the research questions are concerned with changes in a particular organization, or designated group of families, panel design is appropriate. If it is not necessary to follow a particular segment, do a longitudinal. 3. Validity problems posed by the research. The validity of a panel study depends on the researcher’s ability to follow the same people over time.

13. Constructing Questions The first consideration is the manner for collecting the data. • Mail Surveys Must be particularly easy to read and understand, with very clear instructions (since respondents are unable to obtain explanations). • Telephone Surveys Cannot use questions with long lists of response options; respondents may forget the first possible responses.

14. CONSTRUCTING QUESTIONS • Personal Interviews While it is possible for trained interviewers to respond better to the subject when clarification is necessary, the validity of the survey is challenged if clarifications are frequently necessary. Also, respondents might be reluctant to answer sensitive or private questions if in a face to face situations. • Group Administration As many individuals are reluctant to admit their misunderstandings, questions must be clearly worded along with the instructions given to the group.

15. Smith’s Recommendations for Survey Design There are five topics to follow: 1. Determine the types of questions required. 2. Construct questions that will elicit the desired information from respondents. 3. Organize the questions in patterns that maximize chances of getting the required information. 4. Develop appropriate instructions for responding to the questions. 5. Design an attractive presentational format.

16. 1. Determine the types of questions required. • Should you use factual or opionion questions (or both)? • Should you use closed or open questions (or both)?

17. 1. Determine the types of questions required. • Factual questionssolicit information about respondents’ back grounds, behaviors, or habits. • Examples • Demographic data: age, sex, race, marital status, education, income level. • Behavioral data: How many times during a week do you read the news paper. How often do you speak to your supervisor. How many times a semester to you meet with your advisor?

18. 1. Determine the types of questions required. • Opinion questions seek information about a respondent’s motives, intentions, likes and dislikes, attitudes, etc. • Examples • Asking people to express their level of agreement with statements such as: • Newspaper news is more believable than television news. • Sharing intimate information makes my partner happy. • May ask respondents to rank order a set of choices such as in below. • Please rank each of the following as sources of information about your organization: co-workers, supervisors, other managers, top-management.

19. 1. Determine the types of questions required. • Descriptive studies tend towards more factual questions, and analytical studies towards more opinion questions. • Can you answer why?

20. 1. Determine the types of questions required. • Open ended questionsallow subjects to provide their own answers. • Examples • What type of television program do your prefer? (instead of a list of programs) • What are the most effective means your organization uses to communicate benefit information? • How satisfied are you with your interpersonal relationships?

21. 1. Determine the types of questions required. • There are advantages and disadvantages to use of open-ended questions. • Advantages: • 1. In interview formats, researchers can follow up open end questions to get more information. • Why did you give that particular answer? • Could you explain your answer in more detail? • 2. Open-ended questions allow for answers that researchers did not foresee in the construction of the questionnaire. This advantage is particularly important in exploratory studies or pilot studies.

22. 1. Determine the types of questions required. • There are advantages and disadvantages to use of open-ended questions. Disadvantages: 1. Open ended questions are very time consuming. Researchers must categorize responses using techniques such as content analysis. 2. Because there may be many unique answers, the survey may not provide generalizable data. 3. There are often serious difficulties in interpreting the answers to open ended questions.

23. 1. Determine the types of questions required. • There are advantages and disadvantages to use of open-ended questions. Disadvantages: 1. Open ended questions are very time consuming. Researchers must categorize responses using techniques such as content analysis. 2. Because there may be many unique answers, the survey may not provide generalizable data. 3. There are often serious difficulties in interpreting the answers to open ended questions. • For example, consider the responses to the following open-ended question: • Why do you listen to this station? • The station is good, but sometimes it makes me want to vomit. • My kids hate it, so sometimes I turn it up real loud. • My girlfriend forces me to listen.

24. 1. Determine the types of questions required. Closed ended questions require respondents to select answers from a lists of possibilities provided by the researcher. Advantages: 1. Quick analysis. 2. Easily quantified. 3. Uniform responses. Disadvantages: 1. Requires researchers to spend much time in the development and preparation of the survey questionnaire. 2. Subjects may have answers different from those available. • Sometimes taken care of by an “Other”category and a few blank lines. • Better solved through first doing a pilot study with open-ended questions

25. 2.Construct questions that will elicit the desired information from respondents. Survey questions should be clear and unambiguous. • All words and phrases should be read from the point of view of the respondent. • Check to limit, as much as possible, any possible multiple interpretations.

26. 2. Construct questions that will elicit the desired information from respondents. Consider, for example, all the possible interpretations that can result from the following question: Do you regularly disclose intimate information with your partner? regularly? How often is regular? The better choice is to use “Do you each day”. disclose?What does that mean? To disclose is to not close the door? The better choice is to use “tell”. partner? Tennis partner? Business partner? Partner in crime? The better choice is to use “husband/wife” or “boyfriend/girlfriend” or “member of immediate family.” Survey questions should be clear and unambiguous. • All words and phrases should be read from the point of view of the respondent. • Check to limit, as much as possible, any possible multiple interpretations.

27. 2. Construct questions that will elicit the desired information from respondents. • When properly edited, the question • “Do you regularly disclose intimate information with your partner?” • becomes, • Do you each day tell private information about your personal feelings to a member of your immediate family? Survey questions should be clear and unambiguous. • All words and phrases should be read from the point of view of the respondent. • Check to limit, as much as possible, any possible multiple interpretations.

28. 2. Construct questions that will elicit the desired information from respondents. Survey questions should be simple and brief. • The shorter the better. • For example, • Do you periodically find yourself predisposed to divulge secluded information that is otherwise infrequently shared with the majority of your friends to a select few of extraordinarily distinctive receivers? • should be, • Do you often feel like sharing personal information with a few of your friends?

29. 2. Construct questions that will elicit the desired information from respondents. Survey questions should not be leading. • Leading questions contain emotionally loaded language that encourages biased responses. • For example, • Do you agree with the generally accepted view that television programs contain too much explicit violence? • Should be, • Do you think television programming contains too much explicit violence

30. 2. Construct questions that will elicit the desired information from respondents. Survey questions should not be double-barreled. • Doubled barreled questions ask for a single response to multiple questions. • Examples of double-barreled questions: • Do you think the evening news broadcasts is informative and entertaining • Have you stopped kicking your dog lately?

31. 2.Construct questions that will elicit the desired information from respondents. Survey questions should be as non-threatening as possible. • People will often refuse to respond, or respond with socially acceptable responses....even when they are assured of the confidentiality of their responses. • For example, • Do you find yourself sexually excited by certain television programming? • Should be, • Some people find certain television programming to be sexually stimulating. What are your views?

32. 3.Organize the questions in patterns that maximize chances of getting the required information. • Typically, related questions should be grouped together. • Avoids confusing respondents. • Easy to answer questions should be placed first. • Easy questions motivate respondents to continue with a questionnaire. • Closed questions should come first. • Potentially threatening questions should be at the end. • Questions should be ordered to avoid establishing a response bias. • People often repeatedly check on the same side of the scale. Reversing agreement and disagreement can help. Mixing questions of different types also helps.

33. 3.Organize the questions in patterns that maximize chances of getting the required information. • Adopt the appropriate organizational pattern. • Funnel Patterns • Begin with broad questions and progressively focus on more specific ones. • Since respondents are more comfortable answering general questions rather than narrow, use this order when respondents might need encouragement to finish. • Inverted Funnel • Begins with the specific questions and then moves to more general. • Proponents of this approach believe that specific questions are easier to answer. • Proponents also point out that specific questions are less prone to response biases.

34. 4. Develop appropriate instructions for responding to the questions. Instructions should include: • Information about the survey. • Who is conducting the survey, and a brief description of the nature and purpose of the survey. • Assurances that responses will be confidential. • Specific instructions for completing the survey. • Using a bubble sheet? To whom are the results returned? When? • Often helpful to provide some sample questions in the instructions.

35. 5. Design an attractive presentational format. • Must not be cluttered. • Use lots of white spaces... • Must be easy to read. • Avoid small fonts. • Must be neat. • Not only should the survey appear neat, also be sure anyone administering the survey has washed!