1 / 14

How survey design affects analysis

How survey design affects analysis. Susan Purdon Head of Survey Methods Unit National Centre for Social Research. Two general principles. Weighting affects both survey estimates and standard errors

guilloryj
Télécharger la présentation

How survey design affects analysis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. How survey design affects analysis Susan Purdon Head of Survey Methods Unit National Centre for Social Research

  2. Two general principles • Weighting affects both survey estimates and standard errors • Complex sample design (clustering, stratification) affects standard errors but not survey estimates (as long as survey isn’t weighted).

  3. Why weights are used Two main reasons: • Because sample units are selected with non-equal probabilities of selection • To reduce non-response bias

  4. Weighting for non-equal probabilities Non-equal probabilities of selection can be used: • For good statistical reasons: stratification with disproportionate allocation (e.g over-sampling by region) • For convenience: because sampling frame is of larger units than survey units, and need to sub-sample within units (e.g selection of one person per household). • Weights are calculated as the inverse of the probability of selection.

  5. Example 1: WERS98

  6. Effect of selecting one adult per household

  7. Weighting for non-response • Many surveys use weights to reduce non-response bias. • Up-weight low responding groups; down-weight high responding groups.

  8. Weighting for non-response

  9. Effect of weights on estimates • Weighting changes almost all survey estimates (means, percentages, odds ratios, correlation coefficients, regression coefficients etc.)

  10. Unweighted estimate= (25%x38 + 17%x102+15%x27+9%x8) /(38+102+27+8)=20% Weighted estimate= (25%x38x1 + 17%x102x2+15%x27x3+9%x8x4) /(38x1+102x2+27x3+8x4)=18% Effect of weights on estimates

  11. Effect of weighting on standard errors • Standard errors for weighted and unweighted estimates are not the same. • Weighting because of non-equal probabilities of selection tends to increase standard errors. • Weighting for non-response sometimes increases/sometimes decreases ses. Impact tends to be smaller.

  12. Impact of other design features - clustering • Most face-to-face surveys are clustered • Clustering doesn’t change estimates, but it does increase standard errors • Degree of increase depends on cluster size and cluster homogeneity • To account for clustering need to identify the primary sampling unit (psu) on dataset.

  13. Impact of other design features - proportionate stratification • Most surveys use proportionate stratification (either overall or within regions) • Does not affect estimates. Tends to reduce standard errors. Degree depends on choice of stratifiers.

  14. In summary: • To get unbiased estimates need to use survey weights. • To get correct standard errors need to take into account survey design, in particular weighting, clustering and stratification.

More Related