1 / 20

Methodology for Authoring Dialogues

Methodology for Authoring Dialogues. Pamela Jordan University of Pittsburgh Learning Research and Development Center. Agenda. Methodology for authoring dialogues Some lessons learned on authoring computer-mediated dialogues Next steps for projects & discussion.

forrest-velasquez
Télécharger la présentation

Methodology for Authoring Dialogues

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Methodology for Authoring Dialogues Pamela Jordan University of Pittsburgh Learning Research and Development Center

  2. Agenda • Methodology for authoring dialogues • Some lessons learned on authoring computer-mediated dialogues • Next steps for projects & discussion

  3. Authoring preparation methodologies • Corpus-based • Theory-based • Corpus-inspired • Incremental refinement

  4. Corpus-based authoring • Collect corpus of humans interacting on task • Computer mediated • Non-interruptible turns • Analyze for goals/topics & adjust for learning objectives • Analyze goals/topics identified for student responses, look for answer categories of: • Partially correct/incomplete • Partially incorrect • Overly vague • Overly specific • Correct but premature • Identify tutor tactics for each answer category • Analyze student language

  5. Tutoring tactics in ProPl

  6. Form tactics • Pump: can you say more about X? • Hint & reask: fill in a possible missing piece then try again • Socratic: lead through line of reasoning • Simulation: lead through an example & abstract • For additional ones, see chapters 7 & 8 of Evens & Michael (2006), One-on-One Tutoring by Humans and Computers

  7. Applying tactics in ProPl

  8. ProPL student language analysis • Use to define response concepts • Strategy: pick a minimal set of key words that will distinguish between responses

  9. Authoring preparation methodologies • Corpus-based • Theory-based • Corpus-inspired • Incremental refinement

  10. Theory-based authoring • Based on theories about domain/task & learning • Examples of theoretical conceptual tactics: • Definitions & applications of concepts (e.g. distinguish technical & lay senses of terms) • Conceptual variant of a domain principle (e.g. boundary conditions) • Variant of problem

  11. Authoring preparation methodologies • Corpus-based • Theory-based • Corpus-inspired • Incremental refinement

  12. Corpus-inspired authoring • Combination of corpus-based & theory-based • Locate related corpus • Identify theoretical goals • Search for some of those goals within a corpus & refine relative to what can find • Identify theoretically expected student responses • Refine relative to those response instances can find in corpus

  13. Authoring preparation methodologies • Corpus-based • Theory-based • Corpus-inspired • Incremental refinement

  14. Incremental refinement • Author main-path dialogues w/ correct answers • Refine according to answer categories • Author responses to answer categories • Pilot dialogues • Analyze logs & refine authored dialogues

  15. Agenda • Methodology for authoring dialogues • Some lessons learned on authoring computer-mediated dialogues • Next steps for projects & discussion

  16. Recognizing student responses • Language recognizer uses simple technique of minimum edit distance • Minimum edit distance is smallest number of “edits” (insertions/deletions) needed for student response to match a response phrase • For each set of alternative NL phrases (concept) for all responses for the current question, find the minimum edit distance • Select concept with smallest minimum edit distance • If that edit distance is within threshold (default of < .5) then select that concept as the response • Else the student response is not recognized, so follow the unanticipated response path • Beginnings & endings of unmatched parts of responses are not penalized • Stop words (e.g. of, the) are not penalized • Strategy for authoring a response: pick a minimal set of key words that will distinguish between responses for a question

  17. Advice on computer-mediated dialogues • Students prone to refusal to answer e.g., “I don’t know”, “who cares” • Don’t always bottom out • Prod student to try (e.g. “Make your best guess”) • Avoid interrogation: • remember to address coherency; include short recaps, turn and topic transitions, • make some abstractions, meta-information explicit e.g., “Let’s break it down some more”, “First, we’ll identify the givens”. • Assess understanding: • Avoid explicit “do you understand?” • Use trick questions; after success check strength of assertion • “Are you sure?” • “What other forces are there?” (when answer is no more) • Don’t be interactive just for sake of being interactive, instead use it to adapt to individual • Interact in order to diagnose what the student needs • Dialogue slow if cover everything; figure out what can be skipped

  18. Agenda • Methodology for authoring dialogues • Some lessons learned on authoring computer-mediated dialogues • Next steps for projects & discussion

  19. Next steps for projects • Look at dialogue samples/corpus for yur project and identify goals to cover in dialogue • available corpora: http://andes3.lrdc.pitt.edu/TuTalk/corpora/ • For each goal author main path with only correct responses and unanticipated response follow-ups

  20. Discussion & questions • Describe your projects • What help/advice do you anticipate needing?

More Related