Download
1 / 11
110 likes | 236 Vues
This study evaluates the effect of predicting oral reading miscues to improve detection accuracy. By utilizing a reading tutor equipped with a speech recognizer, the project differentiates between common reading errors such as substitutions, omissions, and insertions. Key methodologies include rote prediction and an extrapolative approach, which together enhance the detection of real-word substitutions. The results demonstrate a significant reduction in false alarms and an increase in miscue detection, highlighting the importance of targeted listening in reading instruction.
E N D
Evaluating the Effect of Predicting Oral Reading Miscues Satanjeev Banerjee, Joseph Beck, Jack Mostow Project LISTEN (www.cs.cmu.edu/~listen) Carnegie Mellon University Funding: NSF IERI
Why Predict Miscues? • Reading Tutor helps children learn to read. • Speech recognizer listens for miscues(reading errors) • E.g.: listen for “hat” if sentence to be read has word “hate” • Accurate miscue prediction helps miscue detection.
Real Word Substitutions • Miscues = substitutions, omissions, insertions • Real word substitution = misread target word as another word • E.g. read “hat” instead of “hate” • Most miscues are real word substitutions • ICSLP-02: predicted real word substitutions • Here: evaluate effect on substitution detection
# substitutions detected Substitution detection rate = # substitutions child made 1 1 4 2 How Evaluate Substitution Detection? substitution substitution undetected false alarm substitution detected = # false alarms False alarm rate = = # words correctly read
Evaluation Data • Sentences read by 25 children aged 6 to 10
Rote Method • Uses the University of Colorado miscue database. • For each target word • Sort substitutions by # children who made them. • Predict that the top n substitutions will reoccur, for this word.
Extrapolative Method • Predict the probability that a word is a likely substitution for another word • Pr ( substitution “hat” | target “hate”) • Use machine learning to induce a classifier • Train using University of Colorado miscue database.
Extrapolative Method cont’d Given a target word, predict substitution if Pr ( substitution candidate | target word ) > threshold
Combining Rote and Extrapolative • Aim: Get n substitutions for a given word. • Step 1: Use top n substitutions from rote. • Step 2: If rote predicts k substitutions, k < n, • Then add top n – k substitutions from extrapolative. • Intuition: rote is more accurate, so use when available. If not available, fall back on extrapolative.
Results from Combining Algorithms Truncation = The first 2 to n-2 phonemes of a word – models false starts. [/K AE/ and /K AE N/ for /K AE N D IY/; none for “hate”] Theoretical max = use only those miscues the child actually made.
Conclusion • Evaluated effect on substitution detection of • Two previously published algorithms • A combination of the two algorithms. • Combined approach improved on current configuration (truncations) by • Reducing false alarms by 0.52% abs (12% rel) • Increasing miscue detection by 1.04% (4.2% rel) • Take-home sound byte: Listening for specific reading mistakes can help detect them!
