1 / 18

Predicting Success Using Placement Tests for STEM and Non STEM Math Courses

Predicting Success Using Placement Tests for STEM and Non STEM Math Courses. Michael Pilant , Texas A&M University Jennifer Whitfield, Texas A&M University Robert Hall, Texas A&M University JoungDong Kim, Texas A&M University. Brief History of Texas A&M Math Placement Exam (MPE).

pia
Télécharger la présentation

Predicting Success Using Placement Tests for STEM and Non STEM Math Courses

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Predicting Success Using Placement Tests for STEM and Non STEM Math Courses Michael Pilant, Texas A&M University Jennifer Whitfield, Texas A&M University Robert Hall, Texas A&M University JoungDong Kim, Texas A&M University Scottsdale, AZ

  2. Brief History of Texas A&M Math Placement Exam (MPE) • MPE1 (STEM MPE) consists of 33 algorithmically generated questions covering Pre-Calculus concepts. It was developed, in house, by two lecturers. Subsequently, it was delivered commercially (Cengage->WebAssign->PlaceU). • Before an MPE cutoff was established, students could register for Calc I regardless of MPE score. Based on historical analysis, a cutoff of 22 was established. • Since fitting MPE data in range [0,33] to binary data [pass/fail], is not feasible, we used a logistic curve to relate the probability of passing vs MPE score. • To study retention, and the effect of cutoffs, we also study the cumulative logistic distribution function. • A summer prep/review program for the MPE was established in 2010 • A minimum score of 22 on the MPE was made mandatory in 2011 • MPE2 (NON-STEM MPE) was developed in 2015 • The MPE was un-proctored until Summer 2018 Scottsdale, AZ

  3. Before cutoff established, this shows the pass rate versus MPE score A score of 22 on MPE corresponds to 70% passing, using linear regression. Based on historical data, a cutoff score of MPE=22 was established for enrollment in Calc I beginning in Fall 2011. Scottsdale, AZ

  4. After cutoff implemented, the number of students with MPE<22 is greatly reduced. A score of 22 on MPE still corresponds to 70% passing, using linear regression. The cumulative pass rate, shows the effect of changing the cutoff score (within the class). Scottsdale, AZ

  5. Modeling MPE using Logistic Distribution Function Since Pass/Fail is binary data (Fail=0, Pass=1), a scatterplot of outcome (pass/fail) vs MPE score results in two horizontal lines. Fitting a curve is not feasible … If we consider the number of students passing (or failing) for each MPE score, we are led to analyzing the probability of passiing, for each MPE score. Using actual data, this is modeled by the ratio of NP/(NP+NF). Curve is quite “noisy.” If we want to consider the cumulative pass rate, that is the probability of passing the class with an MPE score greater than or equal to a given score, S, we get the integral of the distribution, which is much smoother (until one reaches the Max MPE=33). Scottsdale, AZ

  6. Calc I Retention Rates, Fall 2000-Fall 2018 Course Changes Algorithmic MPE Cutoff Implemented Increase of 18% over 18 years… Summer PPP Proctored NSF STEP grants Scottsdale, AZ

  7. Development of MPE for NON-STEM Courses Based on the success of the 33 question MPE for Calc I, a 33 question MPE for non-STEM courses was developed. It was increased to 39 questions in 2017. There are two finite math courses (Math 140, 141, 166), two applied calculus (Math 131 and 142), a survey course (Math 167). • Major differences: • STEM courses Calc I, II, III have common exams, non-STEM courses do not. • Calc I has well-defined content, developing an MPE for both Finite Math and Applied Calculus is challenging. • Non-STEM courses can be taken anytime during the students career, and not necessarily in any order (part of core currriculum). Often students wait year they graduate to take math courses! Scottsdale, AZ

  8. Non-STEM course retention rates vs MPE2 Math 141 is a Finite Math course A cutoff of 11 (out of 22) was established for MPE2. This corresponds to a pass rate of 70% on the regression line. Scottsdale, AZ

  9. Non-STEM course retention rates vs MPE2 Math 131 is a Calculus Concepts course A cutoff of 11 (out of 22) was established for MPE2. This corresponds to a pass rate of 70% on the regression line. Scottsdale, AZ

  10. Exam1 Scores in Calc I MPE33 is the 33 question version of MPE1 (for STEM). For this data set, it was un-proctored. The correlation between exam1 and MPE33 is 0.37 (R^2=0.138) Scottsdale, AZ

  11. Exam1 Scores in Calc I MPE16 is the 16 question version of MPE1 (for STEM). It was administered during first week of class, and was proctored. The correlation between exam1 and MPE16 is 0.491 (R^2=0.241) Scottsdale, AZ

  12. Exam1 Scores as an Early Warning Indicator In a previous talk (ICTCM 2017), we analyzed the relationship between exam1, MPE16, MPE33 and final average. Students were defined to be “at risk” if their first exam score was <70% or their computed MPE33 score (MPE16 * 33 / 16) was in the range [0,16] The results of using MPE16 only, and MPE16 along with exam1 scores, are shown In the following slides. Scottsdale, AZ

  13. Predicting At-Risk Students Using Only MPE16 (Given during first week of class) False Positive = 236/541 = 43.62% False Negative = 259/1480 = 17.50% Correctly Identified At Risk = 305/564 = 54.08% Correctly Identified Not At-Risk = 1221/1457 = 83.80% Scottsdale, AZ

  14. Predicting At-Risk Students Using MPE 16 and Exam 1 Scores (Week 6) False Positive = 142/528 = 26.89% False Negative = 178/1493 = 11.92% Correctly Identified At Risk = 386/528 = 68.44% Correctly Identified Not At-Risk = 1315/1457 = 90.25% Scottsdale, AZ

  15. Proctored vs Non-Proctored Results MPE vs SAT Non-Proctored Proctored Scottsdale, AZ

  16. Proctored vs Non-Proctored Results Midterm Grades Scottsdale, AZ

  17. Summary • For STEM course (Calc I and Calc II) • An MPE1 score of 22 out of 33 (on proctored exam) is required (or passing pre-Calculus course) for enrollment • Exams 1, 2, 3 are common exams, across all sections (except honors) • Exam 1 can be retaken if student scores below 70 • Final Exam replaces lowest common exam score • Cutoff for C is now 67%, D is 57% (A, B unchanged at 90, 80) • Summer PPP is available (before first semester) • Week in Review and Help Sessions are available to all students Scottsdale, AZ

  18. Summary • For non-STEM course: • An MPE2 score of 13 out of 26 is required for enrollment • Cutoff for C is now 67%, D is 57% (A, B unchanged at 90, 80) • No common exams • Week in Review and Help Sessions are available to all students • PPP equivalents being developed • Analog of “pre-calculus” for non-stem courses has been developed Scottsdale, AZ

More Related