Collegiate Learning Assessment 2010-11

1. Collegiate Learning Assessment 2010-11 Montclair State University

2. Assessing Learning Outcomes in General Education • The assessment of learning outcomes related to general education is essential for continued accreditation by the Middle States Commission on Higher Education • Standard 12 of Middle States’ “Characteristics of Excellence” asks institutions to provide evidence that their students “demonstrate college-level proficiency in general education” • Standard 14 asks institutions to continuously assess learning outcomes, including general education outcomes • The University has agreed to assess learning outcomes in general education through its participation in the Voluntary System of Accountability (VSA) • The General Education Committee reviewed available instruments, and determined that the Collegiate Learning Assessment (CLA) was most suitable for the University

3. The Instrument • The Collegiate Learning Assessment (CLA) assesses Analytic Reasoning and Evaluation, Writing Effectiveness, Writing Mechanics, and Problem Solving • It relies on constructed-response tasks rather than on multiple-choice items • Students are asked to complete one Performance Task, or a combination of one Make-an-Argument, and one Critique-an-Argument, prompt • Performance Tasks include a document library that students use to answer several open-ended questions about a hypothetical, but realistic, situation • Make-an-Argument prompts ask students to support or reject a position on some issue, while Critique-an-Argument prompts ask students to evaluate the validity of an argument made by someone else

4. CLA Administration Fall 2010 Spring 2011 100 Seniors Tested Sampling Universe: “Native” Seniors with available SAT scores (N=1,125) Invitations to random sample of students with participation voluntary Incentive: $75 Red Hawk Online test in a proctored setting in CADA • 100 Freshmen Tested • Sampling Universe: First-time Freshmen with available SAT scores (N=2,101) • Invitations to random sample of students with participation voluntary • Incentive: $75 Red Hawk • Online test in a proctored setting in CADA

5. Representativeness of Test-Takers • Freshman and Senior test-takers were generally representative of the entire population • Freshman test-takers had higher SAT scores, higher HS GPA, and lower HS rank, but none of the differences were statistically significant • Senior test-takers had lower SAT-M scores, higher SAT-CR and SAT-W scores, and higher college GPA, but only the GPA difference was statistically significant • Differences between test-takers and non-test-takers for both freshman and senior populations were statistically significant for race/ethnicity, but not gender • Differences between test-takers and non-test-takers, by College/School, were statistically significant for seniors, but not for freshmen

6. Means Comparisons

7. Freshman Representativeness

8. Senior Representativeness

9. CLA: Unadjusted Performance Freshman Means Senior Means Total Score: 1177 Performance Task: 1198 Analytic Writing: 1155 Make Argument: 1135 Critique Argument: 1174 SAT: 1013 • Total Score: 1073 • Performance Task: 1033 • Analytic Writing: 1113 • Make Argument: 1113 • Critique Argument: 1110 • SAT: 1026

10. CLA: Value-Added & Estimates Value-Added Score Expected Total Score: Near Performance Task: Above Analytic Writing: Near Make Argument: Near Critique Argument: Near • Total Score: 0.71 • Performance Task: 1.26 • Analytic Writing: -0.11 • Make Argument: -0.30 • Critique Argument: 0.14

11. CLA: Fr. Sub-Score Comparisons

12. CLA: Sr. Sub-Score Comparisons

13. Performing Beyond Expectations

14. Next Steps? • Faculty analysis of results • Development of “actionable” items • Implementation of proposed enhancements • Follow-up assessment of effects of changes