1 / 48

Spoken Programs or Programmers say the darndest things

lok
Télécharger la présentation

Spoken Programs or Programmers say the darndest things

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

    1. Spoken Programs or Programmers say the darndest things! Andrew Begel, Susan L. Graham Computer Science Division, EECS University of California, Berkeley

    2. VL/HCC 2005 Motivation Programmers conventionally use keyboard Long hours at keyboard leads to higher risk of RSI Can a programmer code using speech? Can a computer understand what the developer says? If the user can say anything, this makes it really difficult to understand. Provide motivations for doing this work. Learning about more natural modes of interaction Put why programming by voice on this slide. Introduce programming by voice as a solution. We use conventionalized programmign languages bcause of training and tools. Not the best way to do things, but too hard to change, so instead well modify the Input to the programming language to make it easier for people. (programming ambiguous because it makes the compielr and validation easier). Already existing tools for adapting to programming languages (declaration before def) Given the programming domain Here are the problems we have with them. If the user can say anything, this makes it really difficult to understand. Provide motivations for doing this work. Learning about more natural modes of interaction Put why programming by voice on this slide. Introduce programming by voice as a solution. We use conventionalized programmign languages bcause of training and tools. Not the best way to do things, but too hard to change, so instead well modify the Input to the programming language to make it easier for people. (programming ambiguous because it makes the compielr and validation easier). Already existing tools for adapting to programming languages (declaration before def) Given the programming domain Here are the problems we have with them.

    3. VL/HCC 2005 Programming by Voice Our Goal Find out how developers use code verbally. Use this to develop a naturally verbalizable input form. Build development environment that supports verbal authoring, navigation, modification. Extend conventional compiler analyses to support ambiguities generated by speech. Learn how developers can use voice-based programming, and iterate design. speak whole story, then say were going to talk about the first one. What we want to do: comp, nav, edit speak whole story, then say were going to talk about the first one. What we want to do: comp, nav, edit

    4. VL/HCC 2005 Programming by Voice Challenges Speech is inherently ambiguous. Programming languages and tools were designed to be unambiguous. Speech tools are poorly suited for programming tasks. Programmers are not used to verbal software development. We use conventionalized programmign languages bcause of training and tools. Already existing tools for adapting to programming languages (declaration before def) (programming not ambiguous because it makes the compiler and validation easier). Introduce programming by voice as a solution. Not the best way to do things, but too hard to change, so instead well modify the input to the programming language to make it easier for people.We use conventionalized programmign languages bcause of training and tools. Already existing tools for adapting to programming languages (declaration before def) (programming not ambiguous because it makes the compiler and validation easier). Introduce programming by voice as a solution. Not the best way to do things, but too hard to change, so instead well modify the input to the programming language to make it easier for people.

    5. VL/HCC 2005 Programming by Voice for (int i = 0; i < 10; i++ ) { }

    6. VL/HCC 2005 Current Tools are Awkward! for ( ; ; ) { } Existing tools badly adapt VR to programming (No compiler technology) More explicitly list out loud what the awkwardness is. Requires users to spell identifiers VR finds small words hard to understand. prescriptive way to enter the program once code is in editor, it can no longer be edited by template. code entry is over-stylized no solution for nav and editing Existing tools badly adapt VR to programming (No compiler technology) More explicitly list out loud what the awkwardness is. Requires users to spell identifiers VR finds small words hard to understand. prescriptive way to enter the program once code is in editor, it can no longer be edited by template. code entry is over-stylized no solution for nav and editing

    7. VL/HCC 2005 for (int i = 0; ; ) { } More explicitly list out loud what the awkwardness is. More explicitly list out loud what the awkwardness is.

    8. VL/HCC 2005 for (int i = 0; i < 10; ) { } More explicitly list out loud what the awkwardness is. More explicitly list out loud what the awkwardness is.

    9. VL/HCC 2005 for (int i = 0; i < 10; i++ ) { } More explicitly list out loud what the awkwardness is. not the first person to worry about this stuff, heres what people have to live with now. More explicitly list out loud what the awkwardness is. not the first person to worry about this stuff, heres what people have to live with now.

    10. VL/HCC 2005 How do Programmers Speak Code? 10 programmers read Java code out loud (Begel 05) Graduate students in Computer Science Five knew Java, five did not Five were native English speakers, five were not Five were educated in U.S.A., five were not Read pre-written code into tape recorder As if speaking to a sophomore-level CS undergrad who knows Java, but does not know the program Most programmers spoke the same way

    11. VL/HCC 2005 for (int i = 0; i < 10; i++ ) { } Natural Language Verbalization Didnt have to verbalize parens, semicolons, etc. Didnt have to verbalize parens, semicolons, etc.

    12. VL/HCC 2005 How Do Programmers Speak Code? Think about how to say this Think about how to say this

    13. VL/HCC 2005 How Do Programmers Speak Code? Think about how to say this Think about how to say this

    14. VL/HCC 2005 How Do Programmers Speak Code? Think about how to say this Think about how to say this

    15. VL/HCC 2005 How Do Programmers Speak Code? Think about how to say this Think about how to say this

    16. VL/HCC 2005 How Do Programmers Speak Code? Think about how to say this Think about how to say this

    17. VL/HCC 2005 How Do Programmers Speak Code? Think about how to say this Think about how to say this

    18. VL/HCC 2005 How Do Programmers Speak Code? Think about how to say this Think about how to say this

    19. VL/HCC 2005 How Do Programmers Speak Code? Think about how to say this Think about how to say this

    20. VL/HCC 2005 How Do Programmers Speak Code? Think about how to say this Think about how to say this

    21. VL/HCC 2005 How Do Programmers Speak Code? Think about how to say this Think about how to say this

    22. VL/HCC 2005 How Do Programmers Speak Code? Think about how to say this Think about how to say this

    23. VL/HCC 2005 How Do Programmers Speak Code? Think about how to say this Think about how to say this

    24. VL/HCC 2005 How Do Programmers Speak Code? Think about how to say this Think about how to say this

    25. VL/HCC 2005 How Do Programmers Speak Code? Think about how to say this Think about how to say this

    26. VL/HCC 2005 How Do Programmers Speak Code? Think about how to say this Think about how to say this

    27. VL/HCC 2005 How Do Programmers Speak Code? Think about how to say this Think about how to say this

    28. VL/HCC 2005 How Do Programmers Speak Code? Think about how to say this Think about how to say this

    29. VL/HCC 2005 How Do Programmers Speak Code? Think about how to say this Think about how to say this

    30. VL/HCC 2005 How Do Programmers Speak Code? Think about how to say this Think about how to say this

    31. VL/HCC 2005 How Do Programmers Speak Code? Think about how to say this Think about how to say this

    32. VL/HCC 2005 How Do Programmers Speak Code? Think about how to say this Think about how to say this

    33. VL/HCC 2005 How Do Programmers Speak Code? Think about how to say this Think about how to say this

    34. VL/HCC 2005 How Do Programmers Speak Code? Think about how to say this Think about how to say this

    35. VL/HCC 2005 How Do Programmers Speak Code? Think about how to say this Think about how to say this

    36. VL/HCC 2005 How Do Programmers Speak Code? Think about how to say this Think about how to say this

    37. VL/HCC 2005 Design Tradeoffs

    38. VL/HCC 2005 Spoken Java Semantically identical to Java Syntactically easier to say than Java Design methodology generalizable to any computer language

    39. VL/HCC 2005 A More Natural Way to Code public class Symbol implements Serializable { } Didnt have to verbalize parens, semicolons, etc. Didnt have to verbalize parens, semicolons, etc.

    40. VL/HCC 2005 A More Natural Way to Code public class Symbol implements Serializable { static HashMap hashtbl = new HashMap(); } All punctuation has English equivalents Open Brace, End For Loop Most punctuation is optional Provide verbalization for all abbreviations Relaxed phrasing for better fit with English (int)foo ? cast foo to integer foo = 6 ? set foo to 6 foo[i]++ ? increment the ith element of array foo All punctuation has English equivalents Open Brace, End For Loop Most punctuation is optional Provide verbalization for all abbreviations Relaxed phrasing for better fit with English (int)foo ? cast foo to integer foo = 6 ? set foo to 6 foo[i]++ ? increment the ith element of array foo

    41. VL/HCC 2005 A More Natural Way to Code public class Symbol implements Serializable { static HashMap hashtbl = new HashMap(); } All punctuation has English equivalents Open Brace, End For Loop Most punctuation is optional Provide verbalization for all abbreviations Relaxed phrasing for better fit with English (int)foo ? cast foo to integer foo = 6 ? set foo to 6 foo[i]++ ? increment the ith element of array foo All punctuation has English equivalents Open Brace, End For Loop Most punctuation is optional Provide verbalization for all abbreviations Relaxed phrasing for better fit with English (int)foo ? cast foo to integer foo = 6 ? set foo to 6 foo[i]++ ? increment the ith element of array foo

    42. VL/HCC 2005 A More Natural Way to Code for (int i = 0; i < 10; i++ ) { } Didnt have to verbalize parens, semicolons, etc. Didnt have to verbalize parens, semicolons, etc.

    43. VL/HCC 2005 Too Many Ambiguities for (int i = 0; i < 10; i++ ) { } A human could eyeball this.A human could eyeball this.

    44. VL/HCC 2005 Sometimes Its Non-Obvious fore *= 8; file.tooLode.times = won

    45. VL/HCC 2005 SPEED: Speech Editor Build an editor that supports naturally verbalized programs SPEED: SPEech EDitor Based on Dragon Naturally Speaking, Eclipse IDE, Harmonia Spoken Java Language for Composition Spoken Command language for Navigation, Editing, Template instantiation, Refactorings, Search Visual feedback for training

    46. VL/HCC 2005 Future Study - SPEED Usability Goal: Understand how SPEED can be used by expert programmers Train expert Java programmers on SPEED Author new code Build a Linked List data structure with associated algorithms Modify existing code Change abstraction representation and update algorithms Say this is for expert java programmers. Rephrase system understanding (everything else is about the person, but this is about the system). Find some terminology from AI systems evaluations (false positive rate etc) Say this is for expert java programmers. Rephrase system understanding (everything else is about the person, but this is about the system). Find some terminology from AI systems evaluations (false positive rate etc)

    47. VL/HCC 2005 Metrics User Metrics Task completion time Vocabulary mistakes, Forgotten vocabulary Grammatical mistakes, Grammatical substitutions SPEED Metrics Word tokenization errors Disambiguation errors Irresolvable ambiguities

    48. VL/HCC 2005 Spoken Programs Learn how developers speak code out loud Analyze speech and categorize ambiguities Use study to design naturally verbalizable input for programming Create programming tools designed for spoken interaction Evaluate design and tools by studying programmers using voice for software development fix animation fix animation

    49. VL/HCC 2005 End of Talk

More Related