Designing Surveys Questionnaires

1. Designing Surveys & Questionnaires Reliability, Consistency, Validity and Sampling

2. Surveys & Questionnaires Classic examples The Kinsey Report Michigan Report on Drug Use Monitoring the Future (www.isr.umich.edu/src/mtf) National Longitudinal Survey onYouth (www.bls.gov/nlsfaqs.htm)

3. Questionnaire Design Types of Question Open-ended fill-in-the-blank Closed (Restricted choice) 2-alternative Numerical Range Rating Scale Likert Scale Continuous Scale (Marked) Continuous Scale (Unmarked)

4. Open vs. Closed: Closed May Exaggerate Importance �What is most important problem facing country today?� Energy, schools,abortion, pollution together are mentioned by < 3% of respondents �Which of the following - energy, schools, abortion, pollution - do you think is most important problem, or, if you prefer, you may name a different problem.� Energy, schools, abortion, pollution together are selected by 60% of respondents (After Schuman & Scott, 1987)

5. Open vs. Closed: Open May Diminish Importance �Name one or two especially important events or changes in the past 50 years.� Invention of computer mentioned by <2% of respondents �Which of these events or changes - W W II, space exploration, assassination of JFK, invention of the computer, Vietnam War, seems the most important in the past 50 years, or, if you prefer, you can name an alternative event.� Invention of computer mentioned by 30% of respondents (After Schuman & Scott, 1987)

6. Anchoring Events in Time People do not have easy access to absolute dates: Consider the Memory Questionnaire Place events within defined time frame �How often in the past week have you �?� Mark time frame within meaningful time segments �Since the beginning of the semester, how often have you �?�

7. Memory Questionnaire Name the year in which each movie appeared Gangs of New York Gladiator There�s Something About Mary Mission Impossible (I) Titanic A Beautiful Mind The Matrix Jurassic Park (I)

8. Writing Questions KISS Does the anthropomorphization of animals as substitutes for human characters in cartoons bother you? Be specific, not vague Like, what�s happening? Be neutral, not biased How severely do you beat your wife? Do not ask for multiple answers in one question Do you drink frequently and have sex? Avoid questions with a negation Should the US not support the UN?

9. Questionaire Quality: Reliability Test - Retest Reliability Split Half Reliability Internal Consistency Cronbach�s alpha can be used for all

10. Positive and Negative Affect Scale (PANAS) This scale consists of a number of words that describe different feelings and emotions. Read each item and then mark the appropriate answer in the space next to that word. Indicate to what extent you feel this way right now, that is, at the present moment. Use the following scale to record your answers: 1 2 3 4 5 very slightly a little moderately quite a bit extremely or not at all ___ interested ___ irritable ___ distressed ___ alert ___ excited ___ ashamed ___ upset ___ inspired ___ strong ___ nervous ___ guilty ___ determined ___ scared ___ attentive ___ hostile ___ jittery ___ enthusiastic ___ active ___ proud ___ afraid

11. Scoring the PANAS Positive Negative ___ interested ___ irritable ___ alert ___ distressed ___ excited ___ ashamed ___ inspired ___ upset ___ strong ___ nervous ___ determined ___ guilty ___ attentive ___ scared ___ active ___ hostile ___ enthusiastic ___ jittery ___ proud ___ afraid Sum Pos ___ Sum Neg ___

12. Cronbach�s Alpha Formula: ? = k ( ST2 - ? Sc2 ) k-1 ST2 ST2 = variance for total scores Sc2 = variance for individual test scores

13. Calculating Cronbach�s alpha