tools and analyses for ambiguous input streams n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Tools and Analyses for Ambiguous Input Streams PowerPoint Presentation
Download Presentation
Tools and Analyses for Ambiguous Input Streams

play fullscreen
1 / 85

Tools and Analyses for Ambiguous Input Streams

112 Views Download Presentation
Download Presentation

Tools and Analyses for Ambiguous Input Streams

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. Tools and Analyses for Ambiguous Input Streams Andrew Begel and Susan L. Graham University of California, Berkeley LDTA Workshop - April 3, 2004

  2. Harmonia:Language-aware Editing • Programming by Voice • Code dictation • Voice-based editing commands • Program Transformations • Transformation actions • Pattern-matching constructs LDTA 2004

  3. Harmonia:Language-aware Editing • Programming by Voice • Code dictation • Voice-based editing commands • Program Transformations • Transformation actions • Pattern-matching constructs Human Speech LDTA 2004

  4. Harmonia:Language-aware Editing • Programming by Voice • Code dictation • Voice-based editing commands • Program Transformations • Transformation actions • Pattern-matching constructs Human Speech EmbeddedLanguages LDTA 2004

  5. Harmonia:Language-aware Editing • Programming by Voice • Code dictation • Voice-based editing commands • Program Transformations • Transformation actions • Pattern-matching constructs Human Speech EmbeddedLanguages Each kind of input stream ambiguity requires new language analyses LDTA 2004

  6. for int i equals zero i less than ten i plus plus Speech Example for (int i = 0; i < 10; i++ ) {  } LDTA 2004

  7. Ambiguities for (int i = 0; i < 10; i++ ) {  } 4 int eye equals 0 aye less then10 i plus plus LDTA 2004

  8. Ambiguities ID Spelling? for (int i = 0; i < 10; i++ ) {  } KW or ID? KW or #? 4 int eye equals 0 aye less then10 i plus plus LDTA 2004

  9. for times ate equals zero two plus equals one Another Utterance LDTA 2004

  10. for times ate equals zero two plus equals one Many Valid Parses! 4 * 8 = zero; to += won  for (times; ate == 0; to += 1) {  } fore.times(8).equalsZero(2, plus == 1)  LDTA 2004

  11. Embedded Language Example • C and Regexps embedded in Flex Flex Rule for Identifiers [_a-zA-Z]([_a-zA-Z0-9])*i++; RETURN_TOKEN(ID); LDTA 2004

  12. Embedded Language Example • C and Regexps embedded in Flex Flex Rule for Identifiers [_a-zA-Z]([_a-zA-Z0-9])*i++; RETURN_TOKEN(ID); • Why not this interpretation? [_a-zA-Z]([_a-zA-Z0-9])* i++; RETURN_TOKEN(ID); LDTA 2004

  13. Fortran DO 57 I = 3,10 Legacy Language Example LDTA 2004

  14. Fortran Do Loop DO 57I=3,10 Legacy Language Example LDTA 2004

  15. Fortran Do Loop DO 57I=3,10 DO 57 I = 3 Legacy Language Example LDTA 2004

  16. Fortran Do Loop DO 57I=3,10 Assignment DO 57 I =3 Legacy Language Example LDTA 2004

  17. Fortran Do Loop DO 57I=3,10 Assignment DO57I =3 Legacy Language Example LDTA 2004

  18. Legacy Language Example • PL/I • Non-reserved Keywords IF IF = THEN THEN THEN = ELSE ELSE ELSE = END END LDTA 2004

  19. Legacy Language Example • PL/I • Non-reserved Keywords IF IF = THEN THEN THEN = ELSE ELSE ELSE = END END ID ID KW ID LDTA 2004

  20. Input Stream Classification LDTA 2004

  21. Input Stream Classification Embedded Languages Fall in all Four Categories! LDTA 2004

  22. GLR Analysis Architecture for (i = 0; i < 10; i++ ) {  } Lexer GLR Parser Semantics FOR I FOR ( I LDTA 2004

  23. GLR Analysis Architecture for (i = 0; i < 10; i++ ) {  } Handles syntactic ambiguities Lexer GLR Parser Semantics FOR I FOR ( I LDTA 2004

  24. Our Contribution:XGLR Analysis Architecture for i equals zero ... Lexer XGLR Parser Semantics FOR I FOR I LDTA 2004

  25. Our Contribution:XGLR Analysis Architecture for i equals zero ... Handles input stream ambiguities Lexer XGLR Parser Semantics FOR I FOR I 4 EYE LDTA 2004

  26. = 0 I KW # ID FOR KW LR Parsing Parse Stack Input Stream 1 Parse Table LDTA 2004

  27. = 0 I KW # ID FOR KW LR Parsing Parse Stack Input Stream 1 Parse Table LDTA 2004

  28. = 0 I KW # ID FOR KW LR Parsing Parse Stack Input Stream 1 3 Parse Table LDTA 2004

  29. = 0 I KW # ID FOR KW GLR Parsing Parse Stack Input Stream Parse Table 1 LDTA 2004

  30. = 0 I KW # ID FOR KW GLR Parsing Parse Stack Input Stream Parse Table 1 LDTA 2004

  31. = 0 I KW # ID FOR KW GLR Parsing Parse Stack Input Stream 2 5 Parse Table 1 LDTA 2004

  32. = 0 I # KW ID FOR FOR KW KW GLR Parsing Parse Stack Input Stream 2 4 5 Parse Table 1 3 LDTA 2004

  33. XGLR in Action LDTA 2004

  34. Parsing Homophones 23 FOR BAR LDTA 2004

  35. XGLR Extension: Multiple Spellings, Single and Multiple Lexical Categories FOUR FORE ID 23 FOR BAR KW 4 NUM LDTA 2004

  36. XGLR Extension: Parsers fork due to input ambiguity FOUR 23 FORE ID 23 FOR BAR KW 4 23 NUM LDTA 2004

  37. Each parser shifts its now unambiguous input FOUR 26 23 FORE ID 23 FOR 29 BAR KW 4 35 23 NUM LDTA 2004

  38. The next input is lexed unambiguously FOUR 26 23 FORE ID 23 FOR 29 BAR KW ID 4 35 23 NUM LDTA 2004

  39. ID is only a valid lookahead for two parsers FOUR 26 49 23 FORE ID 23 FOR 29 BAR 42 KW ID 4 35 23 NUM LDTA 2004

  40. Parsing Embedded Languages Example BNF Grammar Contains Languages L and W bL loopLdW ENDL loopL LOOPL |  dW  WHILEW NUMW doW doW  DOW |  L W LDTA 2004

  41. Parsing Embedded Languages Example BNF Grammar Contains Languages L and W bL loopLdW ENDL loopL LOOPL |  dW  WHILEW NUMW doW doW  DOW |  LOOP WHILE 34 END WHILE 56 DO END L W LDTA 2004

  42. LDTA 2004

  43. LDTA 2004

  44. LDTA 2004

  45. LDTA 2004

  46. Parsing Embedded Languages S 0 LOOP WHILE 34 LDTA 2004

  47. S 0 LOOP WHILE 34 Current parse state has ambiguous lexical language LDTA 2004

  48. L 0 S LOOP WHILE 34 W 0 XGLR Extension: Fork parsers, assign one to each lexical language LDTA 2004

  49. L L 0 LOOP KW S WHILE 34 W W 0 LOOP ID XGLR Extension: Single spelling, Multiple lexical categories Lex lookahead both in language L and W LDTA 2004

  50. L L L 0 LOOP 4 KW S WHILE 34 W W 0 LOOP ID Only LOOPL is valid lookahead, and is shifted LDTA 2004