1 / 20

Introduction to Survey Methodology

Introduction to Survey Methodology. Survey Research and Design Spring 2006 Class #2. Today’s Agenda. Introduce the concept of total survey error and Dillman’s tailored design method Discuss possible topics for group projects Divide into groups and get started on project.

luka
Télécharger la présentation

Introduction to Survey Methodology

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Introduction to Survey Methodology Survey Research and Design Spring 2006 Class #2

  2. Today’s Agenda • Introduce the concept of total survey error and Dillman’s tailored design method • Discuss possible topics for group projects • Divide into groups and get started on project Survey Research and Design (Umbach)

  3. What is a survey? (Groves et al., p. 2-3) • A survey is a systematic method for gathering information from a sample of entities for the purposes of constructing quantitative descriptors of the attributes of the larger population of which the entities are members. • More specifically, a survey is information: • Gathered by asking people questions • Collected by having interviewers ask questions, having people read or hear questions and recording own answers • Collected from only a subset of the population rather than from all members Survey Research and Design (Umbach)

  4. Probability sampling • Definition: All sample frame elements have a known, nonzero chance of selection • Simple random • Stratified – sample drawn to control for representation of subgroups of population • Cluster – sample drawn from selected groups Survey Research and Design (Umbach)

  5. Survey Process Define Research Objectives Choose Mode of Collection Choose Sampling Frame Construct and Pretest Questionnaire Design and Select Sample Recruit and Measure Sample Code and Edit Data Make Postsurvey Adjustments Perform Analysis Survey Research and Design (Umbach)

  6. Discussion Questions • Why do we conduct surveys? • In what ways have survey data affected policies and practices? • What are some of the drawbacks or problems associated with survey research? What are some of the problems described by Miller (1995) for the NHSLS? Survey Research and Design (Umbach)

  7. Survey lifecycle (Groves, et al., p.43) Measurement Representation Target Population Construct Measurement Sampling Frame Sample Response Respondents Edited Response Postsurvey Adjustments Survey Statistic Survey Research and Design (Umbach)

  8. Measurement definitions • Construct - Elements of information sought by researcher; broadly defined • Measurement - Concrete ways to gather information about constructs • Response - Data produced in surveys coming from survey measurements • Edited response – review of response (e.g., outlier detection; consistency checks) Survey Research and Design (Umbach)

  9. Representation definitions • Target population - a group of elements or cases that conform to specific criteria; and we intend to generalize the results of the research • Sample frame- a subset of the population; list from which a sample is drawn • Sample - subjects selected from a larger group of people • Respondents – Those successfully measured • Postsurvey adjustments – statistical adjustments made due to nonresponse (e.g., weighting, imputation Survey Research and Design (Umbach)

  10. Quality perspective (Groves, et al., p. 48) Measurement Representation _ Y Target Population m1 Construct Coverage Error Validity _ yc Measurement Yi Sampling Frame Sampling Error Measurement Error _ ys Sample Response yi Nonresponse Error Processing Error _ yr Respondents Edited Response yip Adjustment Error Postsurvey Adjustments _ yrw _ yprw Survey Statistic Survey Research and Design (Umbach)

  11. Observational error • Construct validity • the extent to which the measurement is related to the underlying construct • Measurement error • gap between the ideal measurement and the response obtained • often resulting from poor question working or questions being presented in such a way that inaccurate or uninterpretable answers are obtained • Processing error • gap between the variable used in estimation and that provided by the respondent • coding error, etc. Survey Research and Design (Umbach)

  12. Nonobservational error • Coverage error • Gap between the target population and the sampling frame • The result of not allowing all members of the survey population to have an equal or nonzero chance of being sampled for participation • People that can never be sampled (telephone survey/no telephone) • Exists before the sample is drawn • Sampling error • Gap between the sampling frame and the sample • The result of surveying only some, and not all, elements of the survey population Survey Research and Design (Umbach)

  13. Nonobservational error (continued) • Nonresponse error • Gap between the sample and the respondent pool • People who respond are different than sampled individuals who do not respond in a way relevant to the study • Adjustment error • Statistical adjustments may introduce error in the form of bias or variance Survey Research and Design (Umbach)

  14. Survey error and survey costs • Survey error describes the “deviations from the true values applicable to the population studies”…some describe as quality • When conducting surveys, must balance between cost and error • Survey methodology seeks to identify principles about design, collection, processing and analysis that are linked to quality and costs. In other words, it focus es on the balance between costs and error. Survey Research and Design (Umbach)

  15. Discussion Question Where did we see the balancing of survey error and survey costs in the NHSLS (Miller, 1995)? Survey Research and Design (Umbach)

  16. Tailored Design Method (TDM) • Social exchange theory suggests “that actions of individuals are motivated by the return these actions are expected to bring, and in fact usually do bring, from others.” (Dillman, p. 14). • Tailored design seeks to: • Increase rewards • Reduce costs • Establish trust Survey Research and Design (Umbach)

  17. Increase rewards Show positive regard Say thank you Ask for advice Support group values Give tangible rewards Make questionnaire interesting Give social validation Communicate scarcity of response opportunities Reduce social costs Avoid subordinating language Avoid embarrassment Avoid inconvenience Make short and easy Minimize requests for personal information Emphasize similarity to other requests Establish trust Token of appreciation Sponsorship by legitimate authority Make task appear important Source: Dillman (2000) Survey Research and Design (Umbach)

  18. TDM continued • Exchange concepts must be communicated visually • Takes into account features of the survey situation, so must have knowledge of population, sponsorship, and survey content Survey Research and Design (Umbach)

  19. What problem would you like to address for the project? Survey Research and Design (Umbach)

  20. Group project • What is the problem you plan to address? • What are your research questions? • What constructs do you hope to measure? Survey Research and Design (Umbach)

More Related