1 / 19

Plus ca change: barriers to innovation in survey research

Plus ca change: barriers to innovation in survey research. Patrick Sturgis, University of Southampton and National Centre for Research Methods (NCRM). Paper presented at Government Statistical Service Conference, London, July 6 2011. NCRM.

oberon
Télécharger la présentation

Plus ca change: barriers to innovation in survey research

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Plus ca change: barriers to innovation in survey research Patrick Sturgis, University of Southampton and National Centre for Research Methods (NCRM) Paper presented at Government Statistical Service Conference, London, July 6 2011

  2. NCRM • Network of >10 universities offering an integrated programme of methodological research and training • Quantitative, qualitative and mixed methods • Online archive of papers, videos, presentations, toolkits, etc. • Bursary scheme to support training attendance • Find out more at www.ncrm.ac.uk

  3. Key changes in survey practice • Telephone interviewing • RDD • CAPI • Electronic sample management • Web surveys • Mobile digital devices • Attitude measurement?

  4. 5-point Likert agree/disagree format • Probably the most widely used attitude question format in the UK and throughout the world • BSA, ESS, BHPS, BCS, Taking Part, cohort studies, WVS, EVS, etc. etc…. • But it has long been recognised that there are some major problems with this approach

  5. ‘Working in government statistics is enjoyable’ Strongly agree/agree/neither-nor/disagree/strongly disagree

  6. But there is another problem…

  7. Neither/nor mid-points • Perhaps the most common form of middle alternative is the ‘neither/nor’ in Likert items • Usually treated analytically as implying ‘neutrality’, or being ‘in the middle’ on the issue • But it is also logically consistent with having no opinion • If I do not have an opinion on an issue, it is true that I am neither in favour nor opposed to the statement

  8. Research Design • Face-to-face CAPI interviews on the Ipsos-MORI omnibus quota survey • May 2008 • n=3113 • Respondents randomly allocated to one of three 5-point uni-polar Likert items

  9. Overall, how satisfied or dissatisfied are you with the performance of the European Commission? • 1. Very satisfied • 2. Fairly satisfied • 3. Neither satisfied nor dissatisfied • 4. Fairly dissatisfied • 5. Very dissatisfied

  10. How important or unimportant do you think it is it for Britain to be at the forefront of developments in nanotechnology? 1. Very important 2. Fairly important 3. Neither important nor unimportant 4. Not very important 5. Not at all important

  11. On balance, the advantages of GM foods outweigh any dangers • Strongly agree • Agree • Neither agree nor disagree • Disagree • Strongly disagree

  12. For those selecting ‘neither/nor’: • Which of the following statements best describes why you are neither satisfied nor dissatisfied with the performance of the European Commission? • I don’t really have an opinion on this issue • I have an opinion which is right in the middle on this issue • Neither of the above • If (3) Could you tell me what led you to say you are neither satisfied nor dissatisfied? • PROBE FULLY. For what other reasons? • RECORD VERBATIM

  13. Responses can now be categorised as either: • Substantive (directional) response • Initial Don’t Know • Hidden Don’t Know • Neutral

  14. Table 1 Distribution of middle alternative responses after probing

  15. Reference category = initial DK

  16. Why are survey question formats so resistant to change?

  17. “However, you will have to balance this against the policy needs of your funders; changing the question and/or response categories will mean that the data you collect will not be comparable to other surveys against which you may be benchmarking your data”.

  18. Some predictions for 2020 • General population surveys will routinely be conducted using web data collection • Interviewing conducted by avatars • 5-point agree/disagree questions with neither/nor mid-point still widely used.

  19. Some previous predicitons • Frank Dobson for Mayor of London • West Ham premier league champions 2010

More Related