trindikit a toolkit for flexible dialogue systems n.
Skip this Video
Loading SlideShow in 5 Seconds..
TrindiKit: A Toolkit for Flexible Dialogue Systems PowerPoint Presentation
Download Presentation
TrindiKit: A Toolkit for Flexible Dialogue Systems

TrindiKit: A Toolkit for Flexible Dialogue Systems

135 Vues Download Presentation
Télécharger la présentation

TrindiKit: A Toolkit for Flexible Dialogue Systems

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. TrindiKit: A Toolkit for Flexible Dialogue Systems AI course, spring 2003 Staffan Larsson

  2. This lecture • dialogue modelling in general • architecture & concepts • what’s in TrindiKit? • building a system • extending TrindiKit • feature and advantages of TrindiKit • a sample system: GoDiS

  3. Dialogue modelling • Theoretical motivations • find structure of dialogue • explain structure • relate dialogue structure to informational and intentional structure • Practical motivations • build dialogue systems to enable natural human-computer interaction • speech-to-speech translation • ...

  4. Informal approaches to dialogue modelling • speech act theory (Austin, Searle, ...) • utterances are actions • illocutionary acts: ask, assert, instruct etc. • discourse analysis (Schegloff, Sacks, ...) • turn-taking, pre-sequences etc. • dialogue games (Sinclair & Coulthard,...) • structure of dialogue segments (rather than separate utterances) • can e.g. be encoded as regular expressions or finite automata • qna-game -> question qna-game* answer

  5. Computational approaches implemented in systems and toolkits • finite state automata (CLSU toolkit, Nuance) • frame-based (Philips, SpeechWorks) • plan-based (TRAINS, Allen, Cohen, Grosz, Sidner, ...) • general reasoning (Sadek, ...) • information states (TRINDI: Traum, Bos, ...)

  6. Why build dialogue systems? • theoretical: test theories • e.g. what kind of information does the system need to keep track of? • problem: complex system with many components • practical: natural language interfaces • databases (train timetables etc) • electronic devices (mobile phones,...) • instructional/helpdesk systems • booking flights etc • tutorial systems

  7. What does a system need to be able to do? • speech recognition • parsing, syntactic and semantic interpretation • resolve ambiguities • anaphora and ellipsis resolution, etc... • dialogue management • how does an utterance change the state of the dialogue? • given the current state of the dialogue, what should the system do? • natural language generation • speech synthesis

  8. Why spoken dialogue? • Spoken dialogue is the natural way for people to communicate • computers should adapt to humans rather than the other way around • important to enable system and user to communicate in a natural (human-like) way • mixed initiative • turntaking, feedback, barge-in • handle embedded subdialogues • ...

  9. What’s happening with dialogue systems • Beginning to be used commercially • Limited domains • need to encode domain-specific knowledge; a general system would require general world knowledge • speech recognition is harder with large lexicon • Simple dialogue types • mostly information-seeking • Need to bridge gap between dialogue theory and working systems

  10. What is TrindiKit? a toolkit for building and experimenting with dialogue move engines and systems, based on the information state approach not a dialogue system in itself

  11. Architecture & concepts

  12. control DME module1 module… modulei modulej module… modulen • Total Information State • (TIS) • Information state proper (IS) • Module Interface Variables • Resource Interface Variables resource1 resource… resourcem

  13. Information State (IS) • an abstract data structure (record, DRS, set, stack etc.) • accessed by modules using conditions and operations • the Total Information State (TIS) includes • Information State proper (IS) • Module Interface variables • Resource Interface variables

  14. Dialogue Move Engine (DME) • module or group of modules responsible for • updating the IS based on observed moves • selecting moves to be performed • dialogue moves are associated with IS updates using IS update rules • there are also update rules no directly associated with any move (e.g. for reasoning and planning) • update rules: rules for updating the TIS • rule name and class • preconditon list: conditions on TIS • effect list: operations on TIS • update rules are coordinated by update algorithms

  15. Modules and resources • Modules (dialogue move engine, input, interpretation, generation, output etc.) • access the information state • no direct communication between modules • only via module interface variables in TIS • modules don’t have to know anything about other modules • increases modularity, reusability, reconfigurability • may interact with user or external processes • Resources (device interface, lexicons, domain knowledge etc.) • hooked up to the information state (TIS) • accessed by modules • defined as object of some type (e.g. ”lexicon”)

  16. What’s in TrindiKit?

  17. What does TrindiKit provide? • High-level formalism and interpreter for implementing dialogue systems • promotes transparency, reusability, plug-and-play, etc. • allows implementation and comparison of dialogue theories • hides low-level software engineering issues • GUI, WWW-demo • Ready-made modules and resources • speech • interfaces to databases, devices, etc. • reasoning, planning

  18. TrindiKit contents (1) • alibrary of datatype definitions (records, DRSs, sets, stacks etc.) • user extendible • alanguage for writing information state update rules • GUI: methods and tools for visualising the information state • debugging facilities • typechecking • logs of communication modules-TIS • etc.

  19. TrindiKit contents (2) • A language for defining update algorithms used by TrindiKit modules to coordinate update rule application • A language for defining basic control structure, to coordinate modules • A library of basic ready-made modules for input/output, interpretation, generation etc.; • A library of ready-made resources and resource interfaces, e.g. to hook up databases, domain knowledge, devices etc.

  20. Special modules and resources included with TrindiKit • OAA interface resource • enables interaction with existing software and languages other than Prolog • Speech recognition and synthesis modules • TrindiKit shells for off-the-shelf products, e.g. Nuance • Possible future modules: • planning and reasoning modules • multimodal input and output

  21. Asynchronous TrindiKit • Internal communication uses either • OAA (Open Agent Architecture) from SRI, or • AE (Agent Environment), a stripped-down version of OAA, implemented for TrindiKit • enables asynchronous dialogue management • e.g.: system can listen and interpret, plan the dialogue, and talk at the same time

  22. How to build a system

  23. How to use TrindiKit We start from TrindiKit Implements the information state approach Takes care of low-level programming: dataflow, datastructures etc. TrindiKit information state approach

  24. How to build a basic system Formulate a basic dialogue theory Information state Dialogue moves Update rules Add appropriate modules (speech recognition etc) basic dialoguetheory basic system TrindiKit information state approach

  25. How to build a genre-specific system Add genre-dependent IS components, moves and rules genre-specific theory additions genre-specific system basic dialoguetheory basic system TrindiKit information state approach

  26. How to build an application Add application-specific resources application domain & language resources genre-specific theory additions genre-specific system basic dialoguetheory basic system TrindiKit information state approach

  27. Building a domain-independent Dialogue Move Engine • Come up with a nice theory of dialogue • Formalise the theory, i.e. decide on • Type of information state (DRS, record, set of propositions, frame, ...) • A set of dialogue moves • Information state update rules, including rules for integrating and selecting moves • DME Module algorithm(s) and basic control algorithm • any extra datatypes (e.g. for semantics: proposition, question, etc.)

  28. Specifying Infostate type • the Total Information State contains a number of Information State Variables • IS, the Information State ”proper” • Interface Variables • used for communication between modules • Resource Variables • used for hooking up resources to the TIS, thus making them accessible from to modules • use prespecified or new datatypes

  29. Specifying a set of moves • amounts to specifying objects of type move (a reserved type) • there may be type constraints on the arguments of moves • Example: GoDiS dialogue moves • Ask(Q), Q is a question • Answer(A), A is an answer (proposition or fragment) • Request(),  is an action • Confirm() • Greet • Quit

  30. Writing rules • rule = conditions + updates • if the rule is applied to the IS and its conditions are true, the operations will be applied to the IS • conditions may bind variables with scope over the rule (prolog variables, with unification and backtracking)

  31. Example: a rule from GoDiS rule( integrateUsrAnswer, [ $/shared/lu/speaker = usr, assoc( $/shared/lu/moves, answer(R), false ), fst( $/shared/qud, Q ), $domain : relevant_answer( Q, R ), $domain : reduce(Q, R, P) ], [ set_assoc( /shared/lu/moves, answer(R),true), shared/qud := $$pop( $/shared/qud ), add( /shared/com, P ) ] ).

  32. Building modules • Algorithm • For DME modules: coordinate update rules • For control modules: coordinate other modules • TrindiKit includes a language for writing algorithms • For DME modules: basic imperative programming constructs • For control module: basic imperative constructs plus asynchronous triggers

  33. Sample update algorithm grounding, if $latest_speaker == sys then try integrate, try database, repeat downdate_agenda, store else repeat integrate orelse accommodate orelse find_plan orelse if (empty ( $/private/agenda ) then manage_plan else downdate_agenda repeat downdate_agenda if empty($/private/agenda)) then repeat manage_plan repeat refill_agenda repeat store_nim try downdate_qud

  34. Sample control algorithm (2) input: { init => input:display_prompt, new_data(user_input) => input } | interpretation: { import interpret, condition(is_set(input)) => [ interpret, print_state ] } | dme: { import update, import select, init => [ select ], condition(not empty(latest_moves)) => [ update, if $latest_speaker == usr then select ] } | generation: { condition(is_set(next_moves)) => generate } | output: { condition(is_set(output)) => output } )).

  35. From DME to dialogue system Build or select from existing components: • Modules, e.g. • input • interpretation • generation • output • Still domain independent • the choice of modules determines e.g. the format of the grammar and lexicon

  36. Domain-specific system Build or select from existing components: • Resources, e.g. • domain (device/database) interface • dialog-related domain knowledge, e.g. plan libraries etc. • grammars, lexicons • Example resources: GoDiS VCR control • VCR interface • Domain knowledge • Lexicon

  37. Extending TrindiKit

  38. You can add • Datatypes • Whatever you need • Modules • e.g. General interfaces to speech recognizers and synthesizers • Resources • E.g. General interfaces to (passive) devices • Important that all things added are reasonably general, so they can be reused in other systsems

  39. Datatype definitions • relations • relations between objects; true or false • functions • functions from arguments to result • selectors • selects an object embedded in another object • Operations • Changes the information state

  40. Building modules • DME modules • Specific to a certain theory of dialogue management • Best implemented using rules and algorithms • Other modules • Should be more general, less specific to certain theory of dialogue management • May be easier to implement directly in prolog or other language • TrindiKit algorithm language currently only covers checking and updating the infostate • These modules may also need to interact with other programs or devices

  41. Building resources • Resource • the resource itself; exports a set of predicates • Resource interface • defines the resource as a datatype T, i.e. in terms of relations, functions and operations • Resource interface variable • a TIS variable whose value is an object of the type T • By changing the value of the variable, resources can be switched dynamically • change laguage • change domain

  42. Features and advantages of TrindiKit

  43. TrindiKit features • explicit information state datastructure • makes systems more transparent • enable e.g. context sensitive interpretation, distributed decision making, asynchronous interaction • update rules • provide an intuitive way of formalising theories in a way which can be used by a system • represent domain-independent dialogue management strategies

  44. TrindiKit features cont’d • resources • represent domain-specific knowledge • can be switched dynamically • e.g. switching language on-line in GoDiS • modular architecture promotes reuse • basic system -> genre-specific systems • genre-specific system -> applications

  45. Theoretical advantages of TrindiKit • theory-independent • allows implementation and comparison of competing theories • promotes exploration of middle ground between simplistic and very complex theories of dialogue • intuitive formalisation and implementation of dialogue theories • the implementation is close to the theory

  46. Practical advantages of TrindiKit • promotes reuse and reconfigurability on multiple levels • general solutions to general phenomena enables rapid prototyping of applications • allows dealing with more complex dialogue phenomena not handled by current commercial systems

  47. availability • TrindiKit website • • SourceForge project • development versions available • developer community? • licensed under GPL • more info in • Larsson & Traum: NLE Special Issue on Best Practice in Dialogue Systems Design, 2000 • TrindiKit manual (available from website)

  48. Research goals with GoDiS • explore and implement issue-based dialogue management • starting from Ginzburg’s theory of dialogue semantics based on notion of QUD (Questions Under Discussion) • adapt to dialogue system (GoDiS) and implement • extend theory coverage, taking in relevant theories • general theory of dialogue • minimize effort for adapting dialogue system to new domains • incrementally extending system to handle increasingly complex types of dialogue • clarifies relation between dialogue genres • promotes reuse of update rules • Larsson (2002): Issue-based Dialogue Management (PhD Thesis)

  49. GoDiS: an issue-based dialogue system • Built using TrindiKit • Toolkit for implementing and experimenting with dialogue systems based on the information state approach • Explores and implements issue-based dialogue management • Menu based dialogue • Action-oriented dialogue, VCR application • Multiple tasks, information sharing between tasks • Feedback and grounding • Accommodation, re-raising, clarification • Multi-linguality & mutiple domains