James A Personal Mobile Universal Speech Interface for Electronic Devices
Current Speech Application Concept Phone Client PDA Client Computer Client Speech Application Backend
Current Electronic Devices ??? ??? ??? Speech Application Backend
???Questions??? • History: Why is there a conceptual gap? • Motivation: Is speech a useful modality for “other” electronic devices? • Hardware: How would one get speech in “other” devices? • Architecture: What should the system look like? • Dialog: What should/will these conversations be like?
History • Why is there a conceptual gap? • Speech is still hard. • That will change.
Motivation • Is speech a useful modality for “other” electronic devices? • It seems probable. • There has been some positive research (see Microsoft) • Ideas?
Hardware • How would one get speech in “other” devices? • No need to as long as devices are remote controlled via a known interface. • Refer to system architecture.
Architecture Havi adapter Mobile Speech Client X10 adapter
Dialog • USI Model • Artificial subset language • Tree-structured functions • Universal primitives • User-directed • Great for recognition • Entirely declarative (automatic)
Keywords • hello-james • options • where-am-i, where-was-i • go-ahead, ok • status • goodbye • what-is, what-is-the • how-do-i • more
Session Management • hello-james/goodbye • User: blah blah blah...System: ignoring userUser: hello-jamesSystem: stereo, digital cameraUser: stereoSystem: stereo hereUser: goodbyeSystem: goodbyeUser: blah blah blah...System: ignoring user
Query • what-is path/status • User: what-is-the am frequencySystem: the am frequency is five hundred thirtyUser: what-is randomSystem: random is offUser: what-is-the stereoSystem: the stereo is tuner
help/exploration/implicit navigation • how-do-i.../options/path options • User: control alarm clock radio optionsSystem: alarm, clock, radio, sleep...User: moreSystem: x10, stereoUser: stereo optionsSystem: while turning stereo on: off, am, fm, auxiliary, cd...
invocation/specification/implicit exploration/navigation • Path • User: stereo auxiliarySystem: while turning the stereo on and switching to auxiliary: auxiliaryUser: cdSystem: while switching to cd mode: cdUser: playSystem: while playing a cd: play
list navigation • More • User: radio band am optionsSystem: bracketed list [frequency, kabc, k001, k002, k003, k004, k005][fm][off, volume][alarm, clock, sleep][x10, stereo] rendered: frequency, kabc, k001, k002...User: moreSystem: k003, k004, k005...User: moreSystem: fm, off, volume...
orientation • where-am-i • User: what is the discSystem: the disc is threeUser: where am iSystem: stereo cd disc
Research Questions • Is the subset language learnable? • Once learned, is it efficient? • Are user mistakes infrequent enough? • Are system mistake infrequent enough? • Can one generalize from one device to another? • Is the subset language well retained?
