1 / 16

Missing Data in Research Studies

Missing Data in Research Studies. Joseph A. Olsen. What do I do about missing data?. Introduction. What is certain in life? Death Taxes What is certain in research? Measurement error Missing data Missing data can be:

kalin
Télécharger la présentation

Missing Data in Research Studies

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Missing Data in Research Studies Joseph A. Olsen

  2. What do I do about missing data?

  3. Introduction • What is certain in life? • Death • Taxes • What is certain in research? • Measurement error • Missing data • Missing data can be: • Due to preventable errors, mistakes, or lack of foresight by the researcher • Due to problems outside the control of the researcher • Deliberate, intended, or planned by the researcher to reduce cost or respondent burden • Due to differential applicability of some items to subsets of respondents • Etc.

  4. Some Characteristics of Missing Data • Facets of missing data • Persons • Variables • Occasions • Type of non-response • Unit non-response • Block non-response • Wave non-response • Item non-response • Special non-response problems in longitudinal and clustered data • Attrition/drop-out • Group (e. g. family) member non-response

  5. Missing Data Mechanisms (1) • Preliminaries: • Yobs: The non-missing or observed data • Ymiss: The missing or unobserved data • M: Whether the data on a given item for a given case is missing (1) or not (0) • Missing Completely at Random (MCAR) • The probability that an item is missing (M) is unrelated to either the observed (Yobs) or the unobserved (Ymiss) data • Missing at Random (MAR) • The probability that an item is missing (M) may be related to the observed data (Yobs) but is unrelated to the unobserved data (Ymiss) • Missing Not at Random (MNAR) • The probability that an item is missing (M) is related to the (unknown) value of the unobserved data (Ymiss), even after conditioning on the observed data (Yobs)

  6. Missing Data Mechanisms (2) • The appropriateness of different missing data treatments depends (among other things) on the underlying missing data mechanism • “Real” missing data can seldom be classified into just one of the three (MCAR, MAR, MNAR) • Because we don’t have access to the missing data (Ymiss), we can not empirically test whether or not the data is MNAR • If we know (or can convincingly argue) that the data is not MNAR, a test of whether the data is MCAR is available (e. g. in SPSS Missing Values Analysis).

  7. Missing Data in Research Studies • Missing data mechanism • Missing completely at random (MCAR)—Ignorable • Missing at random (MAR)—Conditionally ignorable • Missing not at random (MNAR)—Nonignorable • Amount of missing data • Percent of cases with missing data • Percent of variables having missing data • Percent of data values that are missing • Pattern of missing data • Missing by design • Missing data patterns • Univariate • Monotonic • File matching • General

  8. Goals of a Missing Data Treatment • Preserve the essential characteristics of the data • Distributions of the variables • Relationships among the variables • Maintain the representativeness of the analyzed data • Provide valid statistical inference (control Type I error) • Maximize the statistical power of the study and its statistical analyses (minimize Type II error) • Avoid bias and instability in the parameter estimates and standard errors for statistical models

  9. Older Missing Data Treatments (1) • Deletion methods • Listwise deletion (complete case analysis) • Pairwise deletion (available case analysis) • Cold deck imputation • Deterministic, logical, or rule-based imputation • Treat missing data for nominal predictors as an additional category • Hot deck (donor case) imputation • Cluster based methods • Distance based (e. g. nearest neighbor) methods • Mean substitution • (Variable) mean substitution • Mean substitution with added random error • Predictor mean substitution with missing data dichotomy

  10. Older Missing Data Treatments (2) • Regression imputation • Regression predicted value imputation • Regression imputation with added random error • Special methods for longitudinal studies and randomized controlled trials • Endpoint only analysis • Last observation carried forward (LOCF) • Intent to treat worst (best) case imputation • Summary growth parameters • Special methods for multi-item scales • Available item method of scale construction • Person mean imputation • Two-way imputation • Two-way imputation with added random error

  11. Newer Missing Data Treatments • Modern state-of-the-art missing data treatments for MAR data • Maximum likelihood • Multiple imputation • Cutting edge investigational missing data treatments for MNAR data • Pattern mixture models • Selection models • Shared parameter models • Inverse probability weighting

  12. Statistical Analysis with Missing Data • What do you get when you don’t specify what you want? What choices do you have within a given analysis procedure? • Often, listwise deletion is the default (and only) option (SPSS Reliability and GLM) • Listwise default with pairwise and mean substitution as options (SPSS Factor and Regression Analysis) • Pairwise default with listwise option (SPSS Correlation) • Modeling approaches that incorporate missing data handling • Survival models • Mixed effects models • Structural equation models • Missing data treatments carried out prior to analysis • Ad hoc methods (Listwise, pairwise, single imputation, etc.) • Modern methods(Maximum Likelihood, Multiple Imputation)

  13. Modern Missing Data Treatments • Maximum likelihood (ML) • Estimates summary statistics or statistical models using all available data • Available in modern structural equation modeling software (Amos, EQS, Lisrel, Mplus, Mx, etc.) • The ML covariance matrix and mean vector can also be obtained from SPSS MVA, and used for standard Regression, Factor analysis, Reliability, and other procedures • There are also freeware and open source programs that can produce the ML covariance matrix and mean vector, usually by using the Expectation Maximization (EM) algorithm (e.g. EMCOV) • Multiple imputation • Imputes individual data values in multiple complete datasets, averaging the results of the statistical analyses across these datasets • Available in the current versions of certain SEM software (Amos, Mplus). • Also available in SPSS (MVA), SAS (Proc MI and MIANALYZE), Stata (mi impute and mi estimate), and stand-alone missing data packages such as SOLAS

  14. Why do social scientists use modern missing data treatments so infrequently? • Lack of awareness or familiarity • They are not convinced of the problems with older methods • The statistical literature on missing data is technically daunting • The techniques aren’t incorporated into the standard statistical analysis procedures used by social scientists • Journal reviewers and editors have not required it

More Related