An Analysis of Online Students: Performance and Differentiation

1. An Analysis of Online Students: Performance and Differentiation AIRPO Conference, Binghamton, NY January 15, 2004

2. By Angel Andreu, Assistant Director Institutional Research Monroe Community College Rochester, New York

3. Outline of Presentation • Background • What precipitated the studies • Initial Research • Secondary Research • Conclusion • Questions/Discussion

4. Background • The SUNY Learning Network (SLN) opened its portal at Monroe Community College in 1997 with thirty-one students. As of Fall 2002, there were 1716 students taking courses online. Figure 1 graphically demonstrates the rapid rise of SLN at MCC

5. Background

6. What Precipitated the Study?

7. What Precipitated the Study?

8. Initial Research Done By • Kathleen Moore, Jeffrey Bartkovich, Marie Fetzner, and Sherrill Ison • Published in Journal of Applied Research in the Community College, Vol.10 No.2 Spring 2003, pp 107 - 118TitledSuccess in Cyberspace: Student Retention in Online Courses

9. Success in Cyberspace “[The] paper attempts to address the relative lack of actual retention data by presenting both archival and survey data on student retention in online courses at a large, comprehensive community college in the Northeast.”

10. Success In Cyberspace Research Questions: • How do retention rates in online courses compare to those in campus-based courses? • What demographic factors are related to retention rates in online courses. • What was the satisfaction level with particular aspects of online learning?

11. Success in Cyberspace Research Questions: • What do students site as reasons for not remaining in or successfully completing online courses? • Does failing to successfully complete one online course impact a student’s willingness to enroll in other online courses in the future?

12. Research Question • How do retention rates in online courses compare to those in campus-based courses and what demographic factors are related?

13. Research Question • What was the satisfaction level with particular aspects of online learning? Experienced students were more satisfied (and first-time students less satisfied) with interaction with faculty, interaction with other students, and directions to get started. All students were dissatisfied with their own performance (not surprisingly, since all students in the survey sample received a grade of “F” or “W”). Satisfaction or dissatisfaction with faculty interaction and with directions to get started in the course are the two areas with the greatest disparity in satisfaction among first-time and experienced students.

14. Research Question • What do students site as reasons for not remaining in or successfully completing online courses? Reasons for not completing online courses varied (see Table 4). “I got behind and couldn’t catch up” was a common reason for all students. More female than male students cited study/work/family balance as a reason, while more male than female students said they lacked motivation. First-time online students were more likely to note problems with course delivery and the online format; but for experienced online students, the reasons tended to focus more on academic issues and personal problems.

15. Research Question

16. Research Question • Does failing to successfully complete one online course impact a student’s willingness to enroll in other online courses in the future? Students’ self-reported likelihood to take another online course in the future was examined in conjunction with other survey responses. Significant correlations (=.01) were found between likelihood to take another online course and the following satisfaction items: • faculty interaction (.63), • online course delivery system in general (.59), • technical help with course (.54), • directions to get started (.54), • directions provided by faculty (.50),

17. Research Question Does failing to successfully complete one online course impact a student’s willingness to enroll in other online courses in the future? • content of the course (.48), • interaction with other students (.45), • and their own performance in course (.39). In addition, likelihood to take another online course was significantly negatively correlated with the following reasons for non- completion: • course was too unstructured (-.54), • didn’t know where to get help (-.32), • and felt too alone, not part of class (-.31).

18. Secondary Research • Factor Analysis • Qualitative Population Logistic Model

19. Factor Analysis • A principle components analysis was used to identify the structural factors underlying the reasons for student withdrawals. The student survey included twenty-two reasons for not successfully completing online classes, ranging from personal motivation to course content to family problems. • In the first study factor analysis was done on the 71 returned surveys. However, another set of surveys were also sent out and analyzed. The factor analysis done was based on both surveys combined for a total sample size of 167.

20. Factor Analysis • The 22 items of the survey clustered into four factors: • F1: “What did I get into?” • F2: Course Design/Professor • F3: Beyond Student Control • F4: Student Behavior

21. Didn’t realize when I registered that it was an online course Lacked basic typing skills Lacked basic computer skills Was able to add another course I wanted more Financial Problems Space opened up in a regular section of the same course Lack of access to a computer Too many technical difficulties Signed up for too many courses/had to cut down course load Course was too difficult Too much reading and writing Not interested in the subject matter Factor Analysis – F1“What did I get into?”

22. Factor Analysis –F2Course Design/Professor • Didn’t line the online format • The online course was too unstructured for me • Felt too alone, not part of a class • Didn’t know where to go for help • Didn’t like the instructor’s teaching style

23. Factor Analysis –F3Beyond Student Control • Personal problems • The course was taking too much time

24. Factor Analysis –F4Student Behavior • Got behind and it was too hard to catch up • Couldn’t handle combined study plus work/family responsibilities • Lack of motivation

25. Qualitative Population Logistic Model

26. An Analysis of SLN Students Performance and Differentiation • Registration Time • Age • Experience • Full-Time/Part-Time • Accuplacer • Brighton Comparison with SLN • Trends

27. Registration Time It was observed that only about 7-8% of high GPA students registered during the first week of classes while 16% of low performing students registered this late. Students who registered five weeks before the start of classes, were less than 10% of the low GPA group, but 17-20% of the high GPA students. Registration peaked in the second week before classes for the successful students whereas the largest registration week for unsuccessful students was the week before classes started. That is after the time when SLN students should have signed onto their course.

28. Registration Time

29. Age

30. Registration Time & Age

31. Experience

32. Accuplacer • Looked at the Accuplacer Reading and Sentence scores to see if there were any significant differences between the reading/sentence scores and SLN GPA of C or better of first year students

33. Accuplacer

34. Brighton Comparison to SLN • One idea in comparing SLN with Brighton is the registration period. In order to do this we count the number of students that are in a category then form an expected count. From this we find the difference between what is observed and expected and divide this difference with the standard error. This new number represents the number of standard deviations an observation is from what is expected.

35. Brighton Comparison to SLN

36. Brighton Comparison to SLN

37. Brighton Comparison to SLN • In sum, there is a distinct pattern of registration between students registering for SLN course versus Brighton courses.

38. Trends

39. Trends • The demographics of MCC enrollment are: younger, full-time, fewer part-time and older students. • So how does this affect the growth of SLN since the population that appears to do poorly is the younger age group and first time full-time?

40. Conclusions • It is the conclusion of this research that we should not expect SLN to parallel the overall growth of the College because the demographics of SLN success are not those predominant in College growth. SLN should target its growth at part-time older students.

41. Conclusions • The later a student registers, the more likely it is that he or she will be in SLN courses. The lack of open class sections on campus late in the registration cycle may be the reason. Extra care should be given to advise late registrants who are young or early in their college career, against registering for SLN courses. When SLN sections of the desired courses are the only ones open, alternative courses should be sought for students with characteristics inappropriate to SLN.

42. Conclusions Low performing SLN students are more likely than successful SLN students to have the following characteristics: • Register in August or later (48% of unsuccessful students vs. 35% of successful students) • Be under 25 years of age (66% of unsuccessful students vs. 43% of successful students) • Be African American (16% of unsuccessful students vs. 7% of successful students) • Be full time with less than 30 earned hours (30% of unsuccessful students vs. 17% of successful)

43. Questions/Discussion

44. Contact Information Angel Andreu aandreu@monroecc.edu MCC Web page: www.monroecc.edu