140 likes | 228 Vues
Understand the concept of generalizability in learning analytics, exploring the factors that affect predictive models when applied to new data sets. Delve into the research of Baker et al. on different types of generalizability and learn about the challenges and advantages of testing for generalizability.
E N D
Learning Analytics: Process & Theory March 24, 2014
Generalizability • Does your model remain predictive when used in a new data set? • Underlies the cross-validation paradigm that is common in data mining • Knowing the context the model will be used in drives what kinds of generalization you should study
What kind(s) of generalizability • Did Baker et al. 2008 look for?
What kind(s) of generalizability • Did Baker et al. 2013 look for?
What kinds of generalizability • Could one test for?
For each kind of generalizability • Has anyone done it? • What are the advantages to doing it? • What are the practical challenges/barriers/costs?
Given… • Given the massive cost of conducting every possible type of generalizability analysis • How do we know when to stop?
Let’s go back to the class on validity… • Since we didn’t quite finish our discussion…
Types of validity we discussed • Ecological • Construct • Predictive • Substantive • Content • Can anyone define and give a quick example of each?
Exercise • In groups of 3 • Write the abstract of the worst EDM paper ever
Exercise #2 • In different groups of 3 • Now write the abstract of the best EDM paper ever