350 likes | 456 Vues
Handbook of Online Research Methods Colloquium 28-29 March 2007. Overview: Online Surveys. Vasja Vehovar University of Ljubljana, Slovenia http://WebSM.org. Introduction. Structure. Computer-assisted survey information collection Key methodological issues Related issues Applications.
E N D
Handbook of Online Research Methods Colloquium 28-29 March 2007 Overview: Online Surveys Vasja Vehovar University of Ljubljana, Slovenia http://WebSM.org
Introduction Structure • Computer-assisted survey information collection • Key methodological issues • Related issues • Applications
Computer-assisted survey information collection Technology and surveys 1930s: applications of probability sampling 1960s: expansion of telephone surveys 1970s: computer technology appears in surveys 1980s: computer-assisted surveying 1990s: Internet mediated surveys
Computer-assisted survey information collection Early CASIC interviewer assisted modes • CATI – Computer-assisted telephone interviewing • CAPI – Computer-assisted personal interviewing • CASI – Computer-assisted self-interviewing
Computer-assisted survey information collection CASIC benefits • Reduced time and costs for data input • Elimination of errors during data transcription • Implementation of advanced features: • automatic skips and branching • randomization of questions and response options • control of answers • inclusion of multimedia elements…
Computer-assisted survey information collection Computerized self-administered questionnaires (CSAQ) Different modes of CSAQ: • disk-by-mail • touch-tone dataentry (TDE) • interactive voice response (IVR) • e-mail surveys • web surveys
Computer-assisted survey information collection Online, Internet, Web, CASIC, CSAQ, CADAC Internet surveys Web surveys
Computer-assisted survey information collection Online, Internet, Web, CASIC, CSAQ, CADAC CSAQ Internet surveys Web surveys
Computer-assisted survey information collection Online, Internet, Web, CASIC, CSAQ, CADAC CSAQ Internet surveys Web surveys CADAC
Computer-assisted survey information collection Interviewer-less and paper-less surveys
Computer-assisted survey information collection Interviewer-less and paper-less data collection
Computer-assisted survey information collection CSAQ input-output technology
Computer-assisted survey information collection CSAQ input-output technology
Computer-assisted survey information collection Technological aspects • Developments of the Internet and related technologies • Importance of broadband Internet access • Possibilities of distribution across various platforms and devices • Fast and easy implementation using specialized software tools
Key methodological issues 1. Probability vs. non-probability sampling • Two major problems of probability samples in Internet surveys: • non-coverage • sampling frame problems • Image of Internet surveys as inherently non-probability ones
Key methodological issues The spurious link Issue of a statistical inference: probability vs. non-probability samples Information-communication technologies Internet survey Management of the survey process Mode of survey data collection
Key methodological issues 2. Sampling and invitation
Key methodological issues 3. Non-response • Generally low and variable response rates to web surveys • Complexity of the response process • Measures to increase response rates: • incentives (e.g. using PayPal) • multiple follow-up contacts • other motivations
Key methodological issues Involvement process Target population Complete respondents Operational population Partial respondents Frame population Starting units Sampled units Solicitation Clickers Absorbed units Aware units Informed units
Key methodological issues Participation in web surveys Respondent’s characteristics Partially under researcher’s control No researcher’s control Researcher-respondent interaction Social environment Technological environment Survey design
Key methodological issues 4. Questionnaire design • Question types and visual elements • Advanced features of computerized questionnaires • Inclusion of multimedia • Problems and considerations: • standardization • impact on respondent’s answers • technical problems measurement error
Key methodological issues 5. Post-survey adjustments • Commonly utilized for correcting: • deviations from probability selections • non-coverage and non-response problems • Importance for non-probability Internet surveys • Promising approaches: • calibration methods • propensity score weighting • modelling and causal analysis
Related issues 1. Costs, errors and management • The managerial process of survey implementation: • costs optimization • data quality • relations between data quality and costs • Cost effectiveness of web surveys: • costs for increasing response rates • evaluations of costs in relation to errors
Related issues 2. Survey mode • Web surveys vs. other modes: • lower response rates (Lozar Manfreda et. al., 2007) • comparable or higher level of validity and reliability (e.g. Fricker et al., 2005; Chang & Krosnick, 2002)
Related issues Mixing survey modes Solicitation Interview PAPI, CAPI, CATI, CASI,… Contact with the respondent personal, telephone, e-mail, mail,… Centralized data management
Related issues Selecting the right mixture Structure of the data (MAR) Willingness to participate Modelling mixed modes Cost models and impact of environment Optimal combination Response rates and behaviour Data quality requirements
Related issues Programing of the questionaire User-friendly interface Standardized questionnaire description in QML CAPI CATI CASI Paper questionnaire Internet survey
Related issues 3. Mixing research methods • Quantitative and qualitative research methods • Flexible combinations of approaches (Morgan, 1998): • qualitative preliminary • quantitative preliminary • qualitative follow-up • quantitative follow-up • New possibilities with the Internet
Related issues Flexible on-line mixed methods (FOMM) Analysis Qualitative method Quantitative method Conclusion Beginning
Related issues 4. Ethics, guides and standards • New ethical dilemmas in survey research: • solicitation process, spam and privacy • data protection • children and minors • combining data… • Development of new standards (e.g. ESOMAR, AoIR, MRA…)
Current and future applications Applications • Different applications according to: • type (internet access panels, on-line voting, intercept…) • organizations (official, academic, commercial) • topics (marketing research, polling opinions, health…) • complexity (simple one-shot surveys vs. complex integrateddata collection)
Current and future applications Related areas • psychological research • online experiments • paradata and online measurement • specific research fields (usability research, HCI...) • emerging areas (e-learning, e-government…)
Current and future applications Trends • Expansion of CASIC modes, options and mixtures • Integration of devices (TV, mobile,..) • Speech recognition and touch-screens • Multimedia and interactivity • Integration with other CADAC (GPS, paradata, observations,...) • Specialisation and professionalisation