1 / 21

Sun-Joo CHO Allan COHEN Brian BOTTGE

Detecting Intervention Effects Using a Multilevel Latent Transition Analysis with a Mixture IRT model. Sun-Joo CHO Allan COHEN Brian BOTTGE. Context.

adina
Télécharger la présentation

Sun-Joo CHO Allan COHEN Brian BOTTGE

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Detecting Intervention Effects Using a Multilevel Latent Transition Analysis with a Mixture IRT model Sun-Joo CHO Allan COHEN Brian BOTTGE

  2. Context • If you are asked by national Department of Education to test the efficacy of a new instructional method which was designed to improve learning disabilities students’ performance. Students’ performance were measured in two time points. • People may concern that • (1) How much progress do student make? • (2) What’s the intervention effects?

  3. (3) What does the progress depend on? (intervention, teacher or both) • (4) How do children’s response process change over time? • (5) Do student perform differently?

  4. Measuring Change • CTT for measuring change • 1. repeated ANOVA or MANOVA: mean change of a total score as a manifest variable across time points • However, the assumption of sphericity (homogeneous and all correlation between any pair of measures equal) is often violated.

  5. LTA-LCM • Latent Transition analysis with Latent Class Model • LTA-LCM assumes that there is no variability on the latent trait within classes. LCM LTA-LCM

  6. LTA-MixIRTM • Latent transition model with a mixture response model Mixture Model LTA_MixIRTM

  7. Mixture Proportion in teacher level latent class Mixture Proportion in individual level latent class Transition proportion IRT model Multilevel LTA-MixIRTM

  8. Proportion of the population in group-level latent class • Proportion of the population in individual-level latent class at Time 1 • Transition proportion for t=2,…,T

  9. Estimation • Assumption: Item parameters are invariant across time and across group-level latent class (item parameters are estimated using post-test data) • Anchor items (scale comparability among latent class?) • Mplus (TWOLEVEL MIXTURE)

  10. Empirical Study Results • 310 students nested under 49 teachers answer 20 items at both pre-test and post-test occasion

  11. (1) Multilevel 1PL ICC= 0.117 (pre-), 0.406 (post-) (2) Detection of Latent Classes A. Only one latent class at pre-test (very few students answer correctly) B. Two latent class at post-test for student-level C. Two latent class at post-test for teacher-level after the latent class at student level were determined

  12. Comparison between multilevel LTA-MixIRTM and LTA-MixIRTM: students were clustered into four transition patterns in multilevel LTA MixIRTM whereas there was two in the LTA MixIRTM.

  13. Do students engage in problem solving differently?

  14. How do students’ response change over time? • On the post-test, 27.4% of the students were classified into student-level Class 2 (pattern 112 and 212). Higher transition proportion of pattern 212 indicate intervention effect.

  15. How much progress? Teachers/intervention? • Pattern 212 has larger mean and large variance, indicating much progress.

  16. In, Pattern 111 (pho=0.856), students tended to retain their relative positions. • In Pattern 211 (pho=0.004), little variability for these student on post-test.

  17. Questions: • 1. How about item parameter recovery? • 2. Anchor item is necessary? • 3. Achilles’ heel in mixture model

More Related