Learning Analytics

1. Learning Analytics NEASC 126th Annual Meeting and Conference December 2011

2. Malcolm Brown, EDUCAUSE Learning Initiative Johann Ari Larusson Brandeis University Dr. Becky Wai-Ling Packard Mt. Holyoke College

3. What’s ahead • Football • What is Learning Analytics? • Examples • Possible benefits • Potential perils and pitfalls

5. Run a play Analyze results Run a play Analyze results Etc.

6. Run a play Analyze results Run a play Analyze results Etc.

7. Run a play Conduct a course Analyze results Analyze evaluations Run a play Conduct a course Analyze results Analyze evaluations Etc. Etc.

8. Learning analytics enable real time interventions

9. What is Learning Analytics? Compilation and analysis of student usage data… …to observe and understand learning behaviors… … to enable appropriate interventions.

10. SIS data Analysis Visualizations LMS data Reports Portfolio data Longitudinal data Alerts Comparative data

11. Audience Instructor facing Student facing

12. Before & after

13. Before & after Before LA What happened?

14. Before & after Before LA After LA What is happening? What happened? What will happen?

15. Examples

16. Social Networks Adapting Pedagogical Practice (SNAPP)

17. Social Networks Adapting Pedagogical Practice (SNAPP) http://research.uow.edu.au/learningnetworks/

19. Bloom’s taxonomy

20. Your questions Is there a way to see when your students are “in” higher order learning modes? Is there a way to see what activities your students are typically doing when in higher order modes?

21. Learner dialog types • Disputational • Cumulative • Exploratory co-reasoning challenging evaluative Reasoned & equitable knowledge sharing

22. Markers

24. Bloom’s taxonomy

25. Learning analytics 1 Uses data (esp. learner-produced data) 2 Performs analysis of that data 3 Discovers information about the learners 4 Enables appropriate intervention

26. Possible benefits • Evidence-based decisions • Measures what students actually do • Identify who needs help • Identify course-wide patterns

27. Potential pitfalls • Privacy • Profiling • Information sharing • Data stewardship • Impoverished model of learning

28. Johann Ari Larusson Brandeis University

29. Point of Originality Example of an R&D Learning Analytics Project

30. The trend/problem • Migration towards larger “gateway” courses • Negative impact on the student’s learning process • Less useful for fostering higher order thinking • Evaluating/monitoring the quality of students’ work • In terms of the depth of the students’ learning • Extremely time consuming, even in smaller classes • Also, the instructor’s work is self-reflective • However • Technology, like blogs, extend the physical boundaries of the classroom, introduce and foster learning communities, even in larger classes • Automatically produce data (electronic form) that can be analyzed • Is in itself a black box but enables us to peek inside the black box

31. Point of Originality • Automated analysis tool: • Via lexical analysis, track students’ language migration from mere paraphrase to mastery • Isolating the moment in time when students demonstrate the ability to explain concepts in their own words, their “point of originality” in time • Recreates the same cognitive activity that educators might ordinarily undergo • Not an automated grading tool • Core components: • WordNet: arranges words by their conceptual-semantic and lexical relationships, notes similarity between two words that don’t have literally identical meanings. • Using input query terms (that relate to key course topics), algorithm calculates how far a student’s language has evolved to explain the course content.

32. Testing the hypothesis • Data: • Co-blogging in 25+ interdisciplinary course (heavy reading list) • Correlating originality scores on blogs and papers to blogging activity • Grades assigned two years earlier by person not involved in the project. • Some results: • More original co-blogging work leads to higher paper grades • Originality variance: • Students with papers above avg. grade have a negative variance. They were at the height of understanding during blogging. • Higher exposure • (to other students’ writing/dialogs) in the blogosphere leads to more original papers • Active participation: • Making more contributions to the blogosphere yields higher paper originality scores

33. Dr. Becky Wai-Ling Packard Mt. Holyoke College

34. Student Retention- Our Coordination Process • Identifying the Key People • Selecting Appropriate Tools • Access • Data Sources

35. Data Sources and Access • Grades and Course Schedule: Advisor • Learning Disability and Mental Health: Dean • Finaid; Transfer Status: Admission • Disciplinary; Dorm Change Requests: Res Life • Current Activities including Sports: Coach • Midsemester Report: Professor, Advisor, Dean • Goals, Accomplishments: Career Center

36. Midsemester Report • Is students’ attendance satisfactory? No • Written work required, turned in, quality acceptable? No • Quality of the student’s work: BORDERLINE Instructor Comments: Jenny, we talked about your absences and missed assignments. As this is a fast-paced course, your absences and missed assignments have added up quickly and have had a negative influence on your overall grade. Please come and see me so we can figure out what is still possible for you.I’d really like to see you succeed.

37. Visual: Key People Initial Faculty Advisor Major Advisor Class Dean Student Profs Career Center Student Advisor (dorm) Pre Prof’l Advisor Coach Student Dept Liaison Coach

38. Discussion