Effective Questionnaire Design Guide

1. Social Research Methods Questionnaire Design

2. References • Oppenheim, A.N. 1978 Questionnaire Design and Attitude Measurement. London: Heinemann • de Vaus, D. A. 1996 Surveys in Social Research, 4th Ed. London: UCL Press (Chapters 6 & 15) • Foddy, W. (1993) Constructing Questions for Interviews and Questionnaires, Cambridge: Cambridge University Press. Plus • Hoinville, G., Jowell, R and associates (1985) Survey Research Practice. London: Gower • Moser, C. A. and Kalton, G (1971) Survey Methods in Social Investigation. Aldershot: Gower

3. When is a questionnaire appropriate? • Have good idea of what to expect • Literate respondents or • Time to administer if not literate • Interested in counts, frequencies etc. • Want to make generalisations to larger population

4. Procedure • Working out the aims of the questionnaire • Selecting the question styles • Designing the questions • Structuring the questionnaire • Piloting the questionnaire • Revising the questionnaire • Administering the questionnaire.

5. Information which can be obtained from questionnaires • Facts and Knowledge (biography) • Actions • Opinions/beliefs • Attitudes • Motives • Past Behaviour • Likely Future Behaviour

6. Choosing the questions • What type of information do you need from this question? • How will the answers serve the aims of the questionnaire? • How will you present the findings from answers to this question? • Do you need numerical scores from the answers?

7. Open questions • What do you think the Government should be doing about global warming?

8. Closed questions 1. What do you think is the most important factor to consider when buying food? Tick just ONE box a) Value for money 1 b) Hygienic processing 2 c) No chemical additives 3 d) Good flavour when cooked 4 e) Easy to prepare 5

9. Open or closed questions?

10. Questionnaire structure Title Introductory paragraph Content questions in sections use filter questions put more difficult questions towards the end do not give away answers to later questions End and thanks

12. Questionnaire layout • Allow space • Instructions (general, section, question, filter question) • Numbers for computer coding • One side of page only (space for comments)

13. Telephone Questionnaires Retention problem = too many categories in answer to remember • use numbered scale • get direction of feeling then intensity • include in question then repeat list • Provide guides for interviewer on questionnaire – distinguish from questions • Make clear what is to be read to respondent and what is not (e.g. instructions for prompt)

14. Checklists and ratings

15. Semantic Differentials How would you rate your mother on these scales? Warmhearted I I I I I Coldhearted Patient I I I I I Impatient Self-assured I I I I I Always worried Unselfish I I I I I Selfish

16. Why pilot a questionnaire? • To expose the questionnaire to new forms of thinking and understanding. • To evaluate the quality of the answers provided by the questionnaire. • To assess and revise the phrasing and content of the questions. • To explore issues of questionnaire structure and question sequence.

17. Use Piloting to check for: • Variation in type of answer • Meaning • Redundancy • Scalability • Non-response (too much) • Acquiescent response (too much agreement with agree/disagree Q) • Flow • Filtering • Timing

18. Scaling Needs to be • Unidimensional • Linear (equal intervals) • Reliable • Valid (esp. for standard scales)

19. Major types of scales • Thurstone • Judges rank agree/disagree items • Lickert • Judges score large list of items • Guttman/Scalogram • Items eliminated to form a ranked scale • Paired comparisons • Pairwise comparing of items

20. Lickert Scales

23. Some problems of question-writing:-Leading questions Are you against giving too much power to trades unions? Why don’t you go more often to the supermarket?

24. Ambiguous questions Is your work made more difficult because you are expecting a baby? Also words like ‘Books’, ‘Dinner’ Multiple-content questions Have you suffered from headaches or sickness lately?

25. Over-complex vocabulary Simple alternatives Avoid big words and technical terms e.g. “Personality”, “Social mobility”, “Anomie”, “Patriarchy”.

26. Over-elaborate phrasing and too complex question Has it happened to you that over a long period of time, when you neither practised abstinence, nor used birth control, you did not conceive? Yes No • vague (what is a ‘long period’ • too factual (‘has it happened to you that …’) • complex - Long sentence and double negative

27. Implicit questions (based on assumptions) • How many children do you have? (Asked of all people) • How old is your car?

28. How often? Use either: • In the last week how often .…? • On average …? • When did you last .…?

29. Embarrassing questions e.g. sexual mores, alcohol consumption, law breaking, taboo subjects (e.g. death, menstruation) Create permissive atmosphere by range of possible answers. e.g. not “Does your child steal?” but, “Which of the following best describes your child? • He or she never steals • He or she sometimes steals (including from home) • He or she has stolen on several occasions.

30. Vague terms e.g. a long time recently near you kind of fairly often on the whole

31. Prestige bias • “Doctors say that …” • “Research has shown that …” • “Is the government right to .…”

32. Patronising tone Avoid this. “Answer these questions to the best of your ability” “‘Dwelling’ means the place where you regularly live”. Use the Sun/Mirror style