Latent Class Analysis

1. Latent Class Analysis Karen Bandeen-Roche October 27, 2016

2. ObjectivesFor you to leave here knowing… • When is latent class analysis (LCA) model useful? • What is the LCA model its underlying assumptions? • How are LCA parameters interpreted? • How are LCA parameters commonly estimated? • How is LCA fit adjudicated? • What are considerations for identifiability / estimability?

3. Motivating ExampleFrailty of Older Adults “…the sixth age shifts into the lean and slipper’d pantaloon, with spectacles on nose and pouch on side, his youthful hose well sav’d, a world too wide, for his shrunk shank…” -- Shakespeare, “As You Like It”

4. The Frailty Construct Fried et al., J Gerontol 2001; Bandeen-Roche et al., J Gerontol, 2006

5. Frailty as a latent variable • “Underlying”:status or degree of syndrome • “Surrogates”: Fried et al. (2001) criteria • weight loss above threshold • low energy expenditure • low walking speed • weakness beyond threshold • exhaustion

6. Part I: Model

7. Latent class model ε1 Y1 Frailty Structural … η Ym εm Measurement

8. Well-used latent variable models General software: MPlus, Latent Gold, WinBugs (Bayesian), NLMIXED (SAS)

9. Analysis of underlying subpopulationsLatent class analysis POPULATION Ui … P1 PJ ∏11 ∏1M ∏J1 ∏JM … … Y1 YM Y1 YM Lazarsfeld & Henry, Latent Structure Analysis, 1968; Goodman, Biometrika, 1974

10. Latent Variables: What?Integrands in a hierarchical model • Observed variables (i=1,…,n): Yi=M-variate; xi=P-variate • Focus: response (Y) distribution = GYx(y/x) ; x-dependence • Model: • Yi generated from latent (underlying) Ui: (Measurement) • Focus on distribution, regression re Ui: (Structural) • Overall, hierarchical model:

11. Latent Variable ModelsLatent Class Regression (LCR) Model • Model: • Structural model: • Measurement model: = “conditional probabilities” > is MxJ • Compare to general form:

12. Latent Variable ModelsLatent Class Regression (LCR) Model • Model: • Measurement assumptions: • Conditional independence • {Yi1,…,YiM} mutually independent conditional on Ui • Reporting heterogeneity unrelated to measured, unmeasured characteristics

13. Latent Variable ModelsLatent Class Regression (LCR) Model • Model: • Measurement assumptions: • Conditional independence • {Yi1,…,YiM} mutually independent conditional on Ci • Reporting heterogeneity unrelated to measured, unmeasured characteristics

14. Analysis of underlying subpopulations Method: Latent class analysis • Seeks homogeneous subpopulations • Features that characterize latent groups • Prevalence in overall population • Proportion reporting each symptom • Number of them = least to achieve homogeneity / conditional independence

15. Latent class analysisPrediction • Of interest: Pr(C=j|Y=y) = posterior probability of class membership • Once model is fit, a straightforward calculation Pr(C=j|Y=y) = = =ij when evaluated at yi

16. Part II: Fitting

17. EstimationBroad Strokes • Maximum likelihood • EM Algorithm • Simplex method (Dayton & Macready, 1988) • Possibly with weighting, robust variance correction • ML software • Specialty: Mplus, Latent Gold • Stata: gllamm • SAS: macro • R: poLCA • Bayesian: winBugs

18. EstimationMethods other than EM algorithm • Bayesian • MCMC methods (e.g. per Winbugs) • A challenge: label-switching • Reversible-jump methods • Advantages: feasibility, philosophy • Disadvantages • Prior choice (high-dimensional; avoiding illogic) • Burn-in, duration • May obscure identification problems

19. EstimationLikelihood maximization: E-M algorithm A process of averaging over missing data – in this case, missing data is class membership.

20. EstimationLikelihood maximization: E-M algorithm • Rationale: LVs as “missing” data • Brief review • “Complete” data • Complete data log likelihood taken as a function of ϕ • Iterate between • (K+1) E-Step: evaluate • (K+1) M-Step: maximize wrt ϕ • Convergence to a local likelihood maximum under regularity Dempster, Laird, and Rubin, JRSSB, 1977

21. EstimationEM example: Latent Class Model

22. EM-AlgorithmLatent class model A process of averaging over missing data – in this case, missing data is class membership. 1. Choose starting set of posterior probabilities • Use them to estimate P and π (M-step) • Calculate Log Likelihood • Use estimates of P and π to calculate posterior probabilities (E-step) • Repeat 2-4 until LL stops changing.

23. Global and Local Maxima Multiple starting values very important!

24. Example: Frailty Women’s Health & Aging Studies • Longitudinal cohort studies to investigate • Causes / course of physical and cognitive disability • Physiological determinants of frailty • Up to 7 rounds spanning 15 years • Companion studies in community, Baltimore, MD • ≥ moderately disabled women 65+ years: n=1002 • ≤ mildly disabled women 70-79 years: n=436 • This project: n=786 age 70-79 years at baseline • Probability-weighted analyses Guralnik et al., NIA, 1995; Fried et al., J Gerontol, 2001

25. 2-Class Model Criterion 3-Class Model CL. 1 “NON-FRAIL” CL. 2 “FRAIL” CL. 2 “INTERMED.” CL. 3 “FRAIL” Weight Loss .073 .26 .072 .11 .54 Weakness .088 .51 .029 .26 .77 .15 .70 Slowness .004 .45 .85 CL. 1 “ROBUST” Low Physical Activity .000 .28 .70 .078 .51 Exhaustion .061 .34 .027 .16 .56 Class Prevalence (P) (%) 73.3 26.7 39.2 53.6 7.2 Example: Latent Frailty ClassesWomen’s Health and Aging Study Conditional Probabilities (π) Bandeen-Roche et al., J Gerontol, 2006

26. 2-Class Model Criterion 3-Class Model CL. 1 “NON-FRAIL” CL. 2 “FRAIL” CL. 2 “INTERMED.” CL. 3 “FRAIL” Weight Loss .073 .26 .072 .11 .54 Weakness .088 .51 .029 .26 .77 .15 .70 Slowness .004 .45 .85 CL. 1 “ROBUST” Low Physical Activity .000 .28 .70 .078 .51 Exhaustion .061 .34 .027 .16 .56 Class Prevalence (P) (%) 73.3 26.7 39.2 53.6 7.2 Example: Latent Frailty ClassesWomen’s Health and Aging Study Conditional Probabilities (π) We estimate that 26% in the “frail” Subpopulation exhibit weight loss” Bandeen-Roche et al., J Gerontol, 2006

27. Part III: Evaluating Fit

28. Choosing the Number of Classes • a priori theory • Chi-Square goodness of fit • Entropy • Information Statistics • AIC, BIC, others • Lo-Mendell-Rubin (LMR) • Not recommended (designed for normal Y) • Bootstrapped Likelihood Ratio Test

29. Entropy Measures classification error 0 – terrible 1 – perfect Ci=j Ci=j Dias & Vermunt (2006)

30. Information Statistics • s = # of parameters • N= sample size • smaller values are better • AIC: -2LL+2s • BIC: -2LL + s*log(N) BIC is typically recommended - Theory: consistent for selection in model family - Nylund et al, Struct Eq Modeling, 2007

31. Likelihood Ratio Tests • LCA models with different # of classes NOT nested appropriately for direct LRT. • Rather: LRT to compare a given model to the “saturated” model • LCA df (binary case): J-1 + J*M • Saturated df: 2M -1 • Goodness of fit df: 2M – J(M+1) P parameters (sum to 1) π parameters (M items*J classes)

32. Bootstrapped Likelihood Ratio Test • In the absence of knowledge about theoretical distribution of difference in –2LL, can construct empirical distribution from data. • per Nylund (2006) simulation studies, performs “best”

33. Example: Frailty Construct Validation Women’s Health & Aging Studies • Internal convergent validity • Criteriamanifestation is syndromic “a group of signs and symptoms that occur together and characterize a particular abnormality” - Merriam-Webster Medical Dictionary

34. Validation: Frailty as a syndrome Method: Latent class analysis • If criteria characterize syndrome: • At least two groups (otherwise, no co-occurrence) • No subgrouping of symptoms (otherwise, more than one abnormality characterized)

35. 2-Class Model Criterion 3-Class Model CL. 1 “NON-FRAIL” CL. 2 “FRAIL” CL. 2 “INTERMED.” CL. 3 “FRAIL” Weight Loss .073 .26 .072 .11 .54 Weakness .088 .51 .029 .26 .77 .15 .70 Slowness .004 .45 .85 CL. 1 “ROBUST” Low Physical Activity .000 .28 .70 .078 .51 Exhaustion .061 .34 .027 .16 .56 Class Prevalence (%) 73.3 26.7 39.2 53.6 7.2 Conditional Probabilities of Meeting Criteriain Latent Frailty Classes WHAS Bandeen-Roche et al., J Gerontol, 2006

36. Results: Frailty Syndrome Validation • Data: Women’s Health and Aging Study • Single-population model fit: inadequate • Two-population model fit: good • Pearson χ2 p-value=.22; minimized AIC, BIC • Frailty criteria prevalence stepwise across classes—no subclustering • Syndromic manifestation well indicated

37. ExampleResidual checking • Frailty construct

38. Part IV: Identifiability / Estimability

39. Identifiability • Rough idea for “non”-identifiability: More unknowns than there are (independent) equations to solve for them • Definition: Consider a family of distributions The parameter is (globally) identifiable iff

40. IdentifiabilityRelated concepts • Local identifiability • Basic idea: ϕ identified within a neighborhood • Definition: F is locally identifiable at if there exists a neighborhood τ about for all τΦ. • Estimability, empirical identifiability: The information matrix for ϕ given y1,…,yn is non-singular.

41. IdentifiabilityLatent class (binary Y) • Latent class analysis (measurement only) • Parameter dimension: 2M -1 • Unconstrained J-class model: J-1 + J*M • Need 2M ≥ J(M+1) (necessary, not sufficient) • Local identifiability: evaluate the Jacobian of the likelihood function (Goodman, 1974) • Estimability: Avoid fewer than 10 allocation per “cell” • n > 10*(2M) (rule of thumb)

42. Identifiability / estimabilityFrailty example • Latent class analysis • Need 2M ≥ J(M+1) (necessary, not sufficient) • M=5; J=3; • 32 ≥ 3∙(5+1) – YES • By this criterion, could fit up to 9 classes • Local identifiability: evaluate the Jacobian of the likelihood function (Goodman, 1974) • Estimability: n > 10*(2M) • n > 10*(25) = 320 - YES

43. ObjectivesFor you to leave here knowing… • When is latent class analysis (LCA) model useful? • What is the LCA model its underlying assumptions? • How are LCA parameters interpreted? • How are LCA parameters commonly estimated? • How is LCA fit adjudicated? • What are considerations for identifiability / estimability?