1 / 34

Online surveys and qualitative analysis: a contradiction in terms?

Online surveys and qualitative analysis: a contradiction in terms?. Martyn Warren. Objectives . Introduce off-the-shelf survey software and its capabilities; Consider the nature of the output from such surveys, and the degree to which some of it can be open to interpretive approaches;

drake
Télécharger la présentation

Online surveys and qualitative analysis: a contradiction in terms?

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Online surveys and qualitative analysis: a contradiction in terms? Martyn Warren

  2. Objectives • Introduce off-the-shelf survey software and its capabilities; • Consider the nature of the output from such surveys, and the degree to which some of it can be open to interpretive approaches; • Stimulate discussion

  3. Perseus - a little bit of magic • Perseus Software Solutions – an off-the-shelf programme facilitating: • Design of questionnaires • Administration of surveys by email/WWW • Collation of responses into MS Access • Calculation of basic descriptive measures • Presentation via Powerpoint • Export into SPSS, Excel, etc

  4. Take a look… • Drafting of questionnaire • Creation of slide presentation • Note textual responses

  5. How many personal computers (i.e. desktop or laptop) does your business use?

  6. My experience • Village webmasters (75% response rate) • Broadband4Devon - small businesses in Devon (38%) • Survey of attitudes of UoP staff to education for sustainable development (29%) • Student attitude survey, module GGX2107 (51%) • Fixing dates for meetings, rehearsals, etc

  7. Discussion in literature • Much interest in late ’90s in methodological issues; • Sampling • Response rates • Ethical issues, including privacy, confidentiality and informed consent • Security • Technology

  8. Discussion in literature (2) • Use of online processes for qualitative approaches: • ‘Virtual ethnography’ (e.g. Hine 2000) – ‘lurking’ around chat rooms, etc. • Online focus groups, Delphi, online interviews (real-time and asynchronous), e.g. Jones (1999) • Content analysis of websites, blogs, etc. (e.g. Technorati, Google)

  9. What about surveys? • Is it possible to create circumstances where survey software can be used to create rich data, susceptible to interpretive analysis? • Implicit assumption in a random selection of methods books that textual data gained through surveys is merely an addendum to quantitative data, to be coded as nominal data and analysed as such.

  10. Surveys in social research • Standardized questions may not get at important aspects of a social group, and “usually there is a limited range of responses a respondent can make”. • Surveys collect data on previously identified variables… limited conceptual flexibility. • Survey may not succeed for cultural or ethical reasons (Williams 2003: 50)

  11. Example – village webmasters Phase 6: Participant triangulation & validation Phases 1 & 2a: Survey of web sites (1073) Phase 4b: Ethno-graphy of online forum (134) Phase 5: Offline ethnography of rural villages (5-10) Phase 3: Survey of web masters (202) Phase 4a: Online probing of webmasters (46) Phase 2b: Offline survey of rural community & development workers (10-15) Warren and Skerratt 2003 online offline

  12. Example question from Phase 2 42. How important do you think the website is to the future of the village? (5-point scale from essential to irrelevant) 43. Can you outline the reasons for your response to Question 42?

  13. Example output from Phase 2 • It is merely an information tool which is available to a minority group of people. I have seen the response to parish council meetings where there have been 2 people from the public attending. If they can not be bothered to attend face to face meetings then what chance has a web page have of having any important value to the village. If this village was less populated (pop. 4400), then the community might feel a little bit closer to one another and the web site might have some higher significance!

  14. Example output from Phase 2 • The village sometimes seems to be full of disinterested sheep-shaggers; always has been and always will be. Until computer monitors have horns and woolly keyboards the Internet will be derided as "a load of rubbish for over educated wastes of space".

  15. Example output from Phase 2 • I derive enormous satisfaction from doing something I enjoy (technically and creatively), from being able to express the love I have for this place to which we moved in 1967 and the friendliness we experienced, and most of all the almost incessant positive feedback - from people casually saying how the maps helped them find their way here, to people catching up with long-lost friends and relatives, tracing the place of their birth, even branches of their family they didn't know existed - one woman was enabled to put to rest the concern over not knowing how her mother had died over 50 years ago. Just last week, a man traced a long-lost pal because he put his surname into Google, and it took him to the Schmoose page to which his pal had contributed - and he's in the middle of China ! Talking about China, one young man e-mailed from an airport in the middle of China to say that he couldn't see his parents in my pictures of the village fete .. on the day it happened .. and! asking were they at home !

  16. Questionnaire, Phase 4 Seven statements posed, arising from Phase 2 responses, each of the form: • Even if everyone in the village had access to the internet, and were able to use it, the website would still only represent/serve a part of the community. (5-point scale, strongly agree to strongly disagree) • Please outline the reasons for your response:

  17. Example output from Phase 4

  18. Example output from Phase 4 • '"Being able to use the Internet" is not the same as feeling that it is the easiest and most natural way to find information and exchange it with banks, government agencies and whatever. There must be at least half of our village community who will never in their lifetimes use the Internet voluntarily for a serious purpose. Even the next generation is going to have a good share of technophobes. We run a great risk of making them a disadvantaged group through no fault of theirs whatever.

  19. Example output from Phase 4 • I am never very keen on "Portal" systems. As the number of possible services and transaction types is enormous and likely to grow exponentially, putting a portal in the way is going to be hell for the portal builder. Not for me thanks! In any case, for the village surfer, what is the point of NOT going directly to the Inland Revenue, the local planning authority, the Dept of Social Security or wherever they want to get to? The government or agency instructions will always say "go to http://junk.agency.gov.uk" and not "go to your village web site and see where they tell you"! In other words I do feel strongly that I could only subtract value from such government services, rather than adding it.

  20. What have we got? • Respondents seem more likely to write at length online than in conventional surveys (evidence?) • Textual responses constrained by pre-specified questions, but many are rich in content. • Researcher-respondent interaction limited, but speed of electronic communication allows some reactive probing. • Lack of the signals – speed, body language, facial expression, voice tone – that can be observed in an interview… • …but at the same time responses are unfiltered by interviewer/transcriber • Reflexivity?

  21. Qualitative researchers… • “…use multiple methods to collect rich, descriptive, contextually situated data in order to seek understanding of human experience or relationships within a system or culture. • “Processes of analytical induction from the data might then lead to the formulation of simple explanatory hypotheses or, using systematic approaches such as grounded theory, the development of complex theories.” (Mann & Stewart, 2000: 2-3)

  22. I would not argue… • … that textual survey data can fill the place of conventional ethnographic techniques… • …but I do think there is a case for taking it more seriously – particularly when arising from online processes – as one source of qualitative data. • What do you think?

  23. Perseus in FSSB • The Faculty has a copy of Perseus Professional in an open access room in Cookworthy • Contact: Jackie Palmer, Learning Technologist, SSB Specialist IT Team, Room 311 Cookworthy. Phone (3)3277 email J.Palmer@plymouth.ac.uk

  24. One alternative (of many)

  25. References (1) • Cho, H., & LaRose, R. (1999). Privacy issues in internet surveys. Social Science Computer Review, 17(4), 421-434. • Crawford, S. D., Couper, M. P., & Lamias, M. J. (2001). Web surveys - Perceptions of burden. Social Science Computer Review, 19(2), 146-162. • Daley, E. M., McDermott, R. J., Brown, K. R. M., & Kittleson, M. J. (2003). Conducting Web-based survey research: A lesson in Internet designs. American Journal of Health Behavior, 27(2), 116-124. • Hine, C. (2000). Virtual Ethnography. London: Sage. • Jones, S. (Ed.). (1999). Doing internet research: critical issues and methods for examining the Net. Thousand Oaks, CA: Sage Publications. • Kaye, B. K., & Johnson, T. J. (1999). Research methodology: Taming the cyber frontier - Techniques for improving online surveys. Social Science Computer Review, 17(3), 323-337. • Knapp, H., & Kirk, S. A. (2003). Using pencil and paper, Internet and touch-tone phones for self-administered surveys: does methodology matter? Computers in Human Behavior, 19(1), 117-134. • Mann, C., & Stewart, F. (2000). Internet communication and qualitative research: a handbook for researching online. London: Sage.

  26. References (2) • Rhodes, S. D., Bowie, D. A., & Hergenrather, K. C. (2003). Collecting behavioural data using the world wide web: considerations for researchers. Journal of Epidemiology and Community Health, 57(1), 68-73. • Schleyer, T. K. L., & Forrest, J. L. (2000). Methods for the design and administration of Web-based surveys. Journal of the American Medical Informatics Association, 7(4), 416-425. • Sills, S. J., & Song, C. Y. (2002). Innovations in survey research - An application of Web-based surveys. Social Science Computer Review, 20(1), 22-30. • Warren, M. F., & Skerratt, S. (2003, 17-19 September 2003). Visiting rural 'virtual villages' through online surveys and online ethnography. Paper presented at the Association of Survey Computing., University of Warwick. • Wellman, B., & Haythornthwaite, C. (Eds.). (2002). The internet in everyday life. Oxford: Blackwell Publishing Ltd. • Witmer, D., Colman, R., & Katzman, S. (1999). From paper-and-pencil to screen-and-keyboard: toward a methodology for survery research on the internet. In S. Jones (Ed.), Doing internet research: critical issues and methods for examining the Net (pp. 299). Thousand Oaks, CA: Sage Publications. • Yeaworth, R. C. (2001). Use of the Internet in survey research. Journal of Professional Nursing, 17(4), 187-193. • Williams, M (2003). Making sense of social research. London, Sage.

More Related