‘Hints for Designing Effective Questionnaires ’

1. ‘Hints for Designing Effective Questionnaires ’ Author: Robert B. Frary Presntor: DeCarlo Bridges

2. Objectives To understand: • Why we use questionnaires • Objectives of a questionnaire • Advantages and disadvantages of questionnaires • Design of questionnaires • Different types of questions used • How to avoid common problems and pitfalls

3. What is a questionnaire? • An instrument to • collect answers to questions • collect factual data – gathers or measures information • A series of written questions / items in a fixed, rational order

4. Effective Questionnaire A well designed questionnaire: • Gives appropriate data which answers your research question • Minimises potential sources of bias • Will more likely be completed

5. Objectives of a questionnaire • To maximise response rate • To ensure that obtained information is accurateand relevant to our study = as simple and as focused as possible

6. Advantages of questionnaires • Can reach a large number of people relatively easily and economically (especially postal/E-mail questionnaires) • Provide quantifiable answers • Relatively easy to analyse

7. Disadvantages of questionnaires • Provides only limited insight into problem • Limited response allowed by questions • Maybe not the right questions are asked • Varying response • Misunderstanding/misinterpretation • Need to get it right first time • Hard to chase after missing data

8. Types of Questionnaire • Self-administrated • By post • E-mail/Internet • Interviewer-administrated • Face to face • Telephone

9. Self-administered questionnaire • Advantages: • Cheap and easy to administer • Preserves confidentiality • Completed at respondent's convenience • Administered in a standard manner • No influence by interviewer

10. Self-administered questionnaire • Disadvantages: • Low response rate • Questions can be misunderstood • No control by interviewer • Time loss

11. Interview-administered questionnaire • Advantages: • Participation by illiterate people • Clarification of ambiguity • Quick answers

12. Interview-administered questionnaire • Disadvantages: • Interviewer bias • Needs more resources • Only short questionnaires possible • Difficult for sensible issues

13. What makes a well designed questionnaire? • Highly structured • Collects the same types of information • Standardized • Allows quantitative and systematic analysis

14. Stages in designing a questionnaire (1) Planning the study: • Decide on goals of study • Know the subject • literature, experts • Formulate a hypothesis • Define information needed to test the hypothesis

15. Stages in designing a questionnaire (2) Determine study population: • Know the respondents • Occupation • Special sensitivities • Education • Ethnic • Language

16. Stages in designing a questionnaire (3) Design questions: • Content of the questions • Format of the questions • Presentation and layout • Codingschedule (if appropriate) • Pilot and refine questionnaire

17. Stages in designing a questionnaire (4) Complete study protocol: • Determine • Cost • Time • Sample size • Response rate

18. Basic Rules (1) • KISSkeep it short and simple • Length of questionnaire: shorter  response rates • Appearance affects • Response rate • Data summarization and analysis •  easy on the eye

19. Basic Rules (2) • Number all items and pages • Put an identifying mark on all pages • Put return address on questionnaire • Put study title in bold on first page • Print directions in bold • Self-addressed envelope!! • Remember the Unique Identifier

20. Basic Rules (3) • Question order • Easy  difficult • General  particular • Factual  abstract • Starting questions • Simple • With closed format • Relevant to main subject • Non-threatening • Neither demographic nor personal questions • Be aware of ordering effects!

21. Basic rules (4) • Group questions by • Topic/ response options • Don’t put most important item last • Questionnaire likely to be completed if • relevant • logical

22. Introduction of questionnaire • Covering letter/introduction for interview • Who you are • Who you work for • Why you are investigating/researching • Where you obtained the respondent’s name • How and where you can be contacted • Absolute guarantee of confidentiality • Length of interview (be honest)

23. Ensuring a High Response Rate • Ensure respondents see its potential and usefulness

24. Content of Questions • Clear focus on research question • Avoid sidetracking • Avoid unnecessary information • Demographic information • Contact information (if non-anonymous)

25. Format of Questions • Adjust to responding audience • Professionals vs. public • Middle class vs. prisoners • Keep sentencessimple and short • Define key words

26. Format of questions Two main question formats • Closed format  give a ‘fixed’ response • Yes, No, Don’t know • Always, sometimes, never • Open format  allow people to express their views in their own words: • What is your most distressing symptom? Please describe: ____________________________________________________________________________________

27. Open or Closed? Closed – forced choice • Advantages: • Simple and quick • Reduces discrimination against less literate • Easy to code, record, analyze • Easy to compare • Easy to report results • Disadvantages: • Restricted number of possible answers • Loss of information • Possible compromise: • Insert field

28. Open or Closed? Open format – free text • Advantages: • Not directive • Allows exploration of issues - generate hypothesis • Used even if no comprehensive range of alternative choices • Good for asking about knowledge and attitudes • Detailed and unexpected answers possible • Disadvantages: • Answer depends on interviewer • Time-consuming • Coding problems • Difficult to analyze! • Difficult to compare groups

29. Problems and Pitfalls • Avoid leading questions • Avoid making questionnaire too long • Typographical / spelling errors

30. Presentation and layout • Using color or printing questionnaire on colored paper may help • Use filter questions, if necessary • Give clear instructions about how to answer the questions

31. Summary A well designed questionnaire: • Will give appropriate data which allow to answer your research question • Will minimise potential sources of bias, thus increasing the validity of the questionnaire • Will much more likely be completed

32. Question 764 …“ CLASS …. . ..HAVE ……ANY ……………Questions?…….. … …….. …… ……… … ………. ……….. …….. ………… ….. …. ……. FINALLY, keep your questionnaire short and the questions simple, focused and appropriate