An Interactionist Approach to Student Success: When Conscientiousness Matters

1. An Interactionist Approach to Student Success: When Conscientiousness Matters Joan R. Poulsen, Deborah A. Kashy, & Gerd Kortemeyer Department of Psychology - Michigan State University www.lon-capa.org Abstract One lesson learned from the person-situation debate is that accurately predicting outcomes involves the interaction between personality characteristics and aspects of the situation. This study examines how the personality trait conscientiousness predicts academic outcomes across a range of course content domains, and estimates the degree to which situational differences in the course structure moderate the conscientiousness-course success relationship. The courses varied in their content domains (e.g., psychology, physics, chemistry), but they all shared a common internet-based homework and course management system: LON-CAPA. We examine how conscientiousness interacts with a variety of course requirements in predicting student performance. Course requirements examined include frequency of homework assignments, percentage of grade that homework contributes, the number of resubmissions (i.e., additional attempts after an incorrect answer is submitted) students were given for typical homework problems. Figure 1. The frequency of homework assignments moderates the effect of conscientiousness on course grade. • Many studies have examined the role of conscientiousness in academic domains. To summarize, people who are higher on conscientiousness, tend to: • Be more likely to complete more years of schooling (Barrick & Mount, 1991; McKenzie & Gow, 2004) • Have higher motivation to perform well (Judge & Ilies, 2002) • Have a positive attitude about school (Heaven et al., 2002) • Have higher self-reported and actual performance at school (Heaven et al., 2002; Paunonen & Ashton, 2001) • Have the tendency to perform more behaviors (e.g., turn in assignments on time) leading to academic success (De Raad & Schouwenburg, 1996). • However, personality is more predictive in some situations than others. Thus, there may be some academic situations in which conscientiousness helps a student more than others. The situation we examine here is how instructors implemented the LON-CAPA course management and assignment system in their classes. • Materials • Characteristics of the Students • School Conscientiousness: Level of conscientiousness in behaviors in the school domain was measured using a modified version of the IPIP scale (alpha = .84). • Grade in course: The final grade in the course was obtained from the registrar’s records on a 0.0 – 4.0 scale. • Percent homework solved: This information was downloaded from the LON-CAPA system and calculated as the number of problems the student solved correctly divided by the number of problems assigned. • GPA: Students’ cumulative grade point average was obtained from the registrar’s records at the end of the term. Figure 2. The amount homework contributes to the course grade moderates the relationship between conscientiousness and course grade. • Characteristics of the Courses • Percent grade that homework counted: We examined the syllabi from each course to determine the percent of the total grade earned from LON-CAPA homework. • Frequency of assignments: Using course syllabi, we assessed how frequently homework assignments were given (e.g., weekly, bi-weekly, etc.). • Number of students in the class: The number of students enrolled in the class at the end of the semester according to the registrar’s records. • Maximum tries: The average number of submissions allowed per problem by the instructor over a course. This information was downloaded from the LON-CAPA system, except in cases of unusual outliers, in which case information from the syllabi were used to determine this value. • What is LON-CAPA? • The LearningOnline Network with a Computer Assisted Personalized Approach • LON-CAPA is an integrated system for online learning and assessment. • It consists of • a learning content authoring and management system allowing new and existing content to be used flexibly • a course management system • an individualized homework and automatic grading system • data collection and data mining system • a content delivery system that will provide gateways to and from NSF's National STEM Digital Library • One reason conscientiousness is interesting to examine in terms of internet-based academic behaviors is the concern that distance education and online components of classes are too unstructured to promote success for many students (Wilkinson & Sherman, 1990; Hiltz, 1994; Guernsey & Young, 1998). With the growing popularity of the use of internet technology in education, it is useful to understand how the structure of a course can impact student performance, and what role conscientiousness plays in this. • Given this previous work, students who are higher on conscientiousness should generally have better grades than students lower in conscientiousness. Also, when courses are structured more loosely (e.g., students are less accountable for their progress), students higher in conscientiousness should perform better than those who are lower on conscientiousness. • In this study, we investigated the following questions: 1. Does conscientiousness predict student success? 2. Does the structure of the online component of a class influence student success? 3. Do students who are more conscientious tend to do better (or worse) if courses are structured in certain ways? 4. How do student-level conscientiousness ratings and course-level differences in class structure interact to predict student performance? Method: Participants 3394 undergraduate students (51% women; 85% Caucasian, 5% African-American, 7% Asian American) enrolled in one of 34 classes at Michigan State University that were using the LON-CAPA system in either Fall semester of 2003 or Spring semester of 2004. Procedure These data were collected as part of a larger study assessing the educational effectiveness of the LON-CAPA system and internet technology. Students in the classes that used LON-CAPA were surveyed at the start and end of the semester. We assessed their level of conscientiousness, attitudes about the LON-CAPA course management system. We downloaded information from the LON-CAPA system about students’ online behaviors. We also obtained information from the registrar and instructors about students’ grades and GPA, as well as information from syllabi about class policies, grading procedures, and how the instructor used the software in his or her class. Similarly, the relationship between conscientiousness and the percent of homework students solved was moderated by three factors: • When homework counted less towards the course grade, students higher on conscientiousness did better on the homework. • When the class size was larger, conscientiousness was more strongly related to the amount of homework students completed. • When students were given more tries to get problems correct, conscientiousness was stronger predictor of how much homework students got correct. • The overall pattern suggests that when a class is structured with fewer external motivational factors, conscientiousness plays a larger role in predicting student success. • The LON-CAPA software provides instructors with a common, scalable platform to assist in all aspects of teaching a course, from lecture preparation to administration of homework assignments and exams. • It provides a sophisticated assignment engine that can create unique homework assignments and exams for each student in a class. Its formative and summative assessment tools grade a broad variety of objective problems and assist in evaluation of essays. • It provides prompt feedback for students and instructors, as well as statistical information on performance and effectiveness of materials. Discussion pages attached to every homework assignment encourage communication among students and faculty. • The LON-CAPA software is freely available and free (GNU General Public License), and may be modified and adapted. Results Sample characteristics Students had a mean conscientiousness rating of 5.13 (SD = 0.88) on a seven-point scale, and an average cumulative GPA of 3.13 (four-point system) (SD =.57). Conscientiousness predicts student success Regression analyses were used to analyze the degree to which school conscientiousness predicted outcomes for students. We controlled for students’ overall GPA and found that higher levels of school conscientiousness predicted: • Higher course grades (b = .06 ***) • Higher percent of homework solved (b = 2.039***) • Fewer guesses on the homework (b = -.884***) Moderators of the relationship between Conscientiousness and success Multi-level regression was used to analyze how different course-level characteristics (Level 2) moderated the effects of conscientiousness (Level 1) on student outcomes. The association between conscientiousness and course grade is moderated by two main factors of interest: • Frequency homework was due: Conscientiousness had a stronger effect on course grade when homework was less frequently assigned. (Figure 1) • Percent of grade homework counted: Conscientiousness had a stronger effect on course grade when homework counted less towards the course grade. (Figure 2) Conclusions The present research suggests that: • Students higher on conscientiousness tend to perform better in university courses. • Students high on conscientiousness perform behaviors that improve their grade (they complete more homework, guess less). • But, conscientiousness may be more predictive of student success in courses that are less rigidly structured. • When students have less external motivation for doing homework, conscientiousness is more important for success. • Also, when students are less accountable (e.g., higher class size, homework is optional), those higher on conscientiousness may have the advantage. • Although conscientiousness predicts student success in college-level courses, the way in which courses are structured influences the strength of this association. • LON-CAPA was developed by an enthusiastic group of faculty and professionals who recognize that Information Technology can play an important role in improving students’ learning and understanding. • Current members of the project include faculty from universities, colleges, and K-12 schools in the US, Canada, Asia, Africa, and Europe. The core development group is located at Michigan State University. • Today LON-CAPA is serving over 16,000 course enrollments per semester at MSU alone, and well over 23,000 course enrollments system-wide, ranging from middle school to graduate level courses. • Disciplines include astronomy, biology, business, chemistry, civil engineering, computer science, family and child ecology, geology, human food and nutrition, human medicine ,mathematics, medical technology, physics, and psychology. The Effects of Conscientiousness in Educational Settings • Previous work on conscientiousness suggests that it positively predicts many performance-related outcomes, such as job performance, and academic behavior.