1 / 20

Impact of Novice Programmer Confusion on Achievement

Explore the relationship between novice programmer confusion and achievement using affective states research. Data collected from 149 students in CS21a, labeled and analyzed with RapidMiner. Results show the effects of confusion on student performance. Ongoing work seeks to automate confusion detection in CS learning environments.

krysta
Télécharger la présentation

Impact of Novice Programmer Confusion on Achievement

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Exploring the Relationship Between Novice Programmer Confusion and Achievement By: Diane Marie Lee Ma. Mercedes Rodrigo Ryan Baker Jessica Sugay Andrei Coronel

  2. Affective States and Achievement • Recent studies have illustrated the relationships between affective states and achievement • Negative affective states have negative impact on student’s achievement (Craig et al, 2006; Rodrigo, 2009; Lagud, 2010)

  3. Confusion • Double-edged/ Dual Nature (D’Mello 2009) • Harmful • Helpful

  4. Goal • Discovery-with-models approach to finding the relationship between novice programmer confusion and achievement

  5. Data Collection • 149 students enrolled in CS21a – Introduction to Computing I • Four lab sessions • BlueJ IDE • BlueJ Plug-in (Jadud and Henriksen, 2009)

  6. Data Collection • Compilation logs = all submissions made to the compiler • Compilation logs include • Computer number • Timestamp • Code • Error message (if any) • And many more!

  7. Data Collection • Total of 340 student-lab sessions • Total of 13,528 compilation logs collected

  8. Data Labeling • Sorted the compilations by student and by Java class name • Grouped the compilations into clips • Clips = 8 compilations • Total: 2,386 clips • Raters were asked to label a sample of 664 clips

  9. Data Labeling • Used low-fidelity text replays • Maintains good inter-rater reliability and efficient in aiding coders to label student disengagement (Baker et al. 2006) • Labels • Confused • Not Confused • Bad Clip • Cohen’s Kappa between raters: 0.77

  10. Data Labeling • Filter out “bad clips” • Remove clips where raters disagreed on the label • Left with 418 clips for model construction

  11. Model Construction • Used RapidMiner version 5.1 • Used J48 Decision Trees • Features were mined from the clips

  12. Model Construction • Feature set used: • Average time between compilations • Maximum time between compilations • Average time between compilations w/ errors • Maximum time between compilations w/ errors • Number of compilations w/ errors • Number of pairs consecutive compilations ending w/ the same error

  13. Kappa: 0.86

  14. Data Relabeling • Model was coded as a Java program • Had the program relabel all the 2,386 clips • Generated three sets of confused-not confused sequences • Correlated the percentage of the sequences of each student to their midterm exam scores

  15. Results

  16. Results

  17. Conclusion • Prolonged confusion has a negative impact on student’s performance • Resolved confusion has a positive impact on student’s performance • A certain amount of confusion is needed for learning

  18. On-going Work • Support the incorporation of tools for automatic detection of confusion in computer science learning environments • Redoing the sampling and clipping method

  19. Thank you Questions?

More Related