1 / 53

CYK Parser

CYK Parser. Von Carla und Cornelia Kempa. Overview. C ocke Y ounger K asami -method. Recognition phase. Example grammar. Number(s)  Integer | Real Integer  Digit | Integer Digit Real  Integer Fraction Scale Fraction  . Integer Scale  e Sign Integer | Empty

dacey-buck
Télécharger la présentation

CYK Parser

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CYK Parser Von Carla und Cornelia Kempa

  2. Overview

  3. Cocke Younger Kasami -method

  4. Recognition phase

  5. Example grammar • Number(s)  Integer | Real • Integer  Digit | Integer Digit • Real  Integer Fraction Scale • Fraction  .Integer • Scale  e Sign Integer | Empty • Digit  0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 • Empty  ɛ • Sign  + | -

  6. Example Sentence: 32.5e+1 • 1. concentrate on the substrings of the input sentence

  7. Building the recognition table

  8. 32.5e +1 is in the language • What problems can we already see in this example?

  9. Another complication: Ɛ- rules Input : 43.1

  10. The ɛ- Problem Shortest substrings of any input sentence : ɛ-substrings We must compute Rɛ the set of non-terminals that derive ɛ Rɛ = { Empty, Scale }

  11. Non- empty substrings of the input sentence • Input : z = z1 z2 z3 z4 ….zn • Compute the set of Non-Terminals that derive the substring of z starting at position i, of length l.

  12. Terminology (also on the handout) • i index we are starting at • l length of this substring • R s i,l set of Non-Terminals deriving the substring s i, l • S i, 0 = ɛ • Set of Non- Terminals that derive ɛ : R s i,0 = R ɛ

  13. S i, l =z i z i+1 …… z i+ l-1

  14. The set of Non- Terminals deriving the substring s i, l : R si, l 1.) substrings of length 0 S i, 0 = ɛ and R si, l = Rɛ 2.) short substrings 3.) longer substrings (say l = j ) All the information on substrings with l < j is available

  15. Check each RH-side (Right-Hand -side) in the grammar to see if it derives s i, l • L  A1 ….Am S i, l ( divided into m segments (= possibly empty)) A1  first segment of s i, l A2  second segment of s i, l …. ….

  16. A 1 ….Am  s i,l • So A1  first part of s i,l (let´s say A1 has to derive a first part of s i, l of length k) A1  s i, k A1 is in the set R s i,k

  17. A 1 ….Am  s i,l • Assuming this A2…Am has to derive the rest: A2 … Am  Si+k, l-k This is attempted for every k

  18. Problems with this Approach 1) Consider A2…Am m could be 1 and A1 a Non-terminal  We are Dealing with a unit- rule A1 must derive the whole substring s i, l and thus be a member of R s i, l But that´s the set we are computing right now …

  19. Solution to this problem • A1  s i, l • Somewhere along the derivation there must be a first step not using a unit rule A1  B … C * s i, l C is the first Non-Terminal using a non-unit-rule in the derivation

  20. Solution cont. At some stage C is added to Rs i, l If we repeat the process again and again At some point B will be added and in the next step A1 will be added  We have to repeat the process again and again until no new Non-Terminals are added to R s i,l

  21. Problem 2 Ɛ-rules Consider all but one of the At derive Ɛ B  A1 A2 A3 A4 A5 …. At B and A1 - t are Non-Terminals A2 – At derive Ɛ So what stays is : B  A1 A unit-rule

  22. We have computed all the Rs i,l • If S is a member of Rs 1, n the start symbol derives z (=s 1, n) (the input string)

  23. CYK recognition with a grammar in ****- form: • What are the Restrictions we want to have on our grammar ?

  24. Useful Restrictions • No ɛ- rules • No unit-rules • Limit the length of the right- hand side of each rule, say to two • What we get out of this: • A  a • A  BC • Where a is a terminal and ABC are Non- Terminals

  25. Chomsky-Normal-Form… (… not only to annoy students ) • Perfect grammar for CYK

  26. How CYK works for a grammar in CNF • Rɛ is empty • R s i, 1 can be read directly from the rules (A  a) A rule A  BC can never derive a single terminal

  27. Procedure • Iteratively (as before) : • 1) Fill the sets R s, 1 directly • 2) Process all substrings of length 1 • 3) Process all substrings of length 2 • 4) Process all substrings of length l • For the first step we use the rules of the form A  a • For all the following steps we have to use the rules of the form: A  BC

  28. CYK and CNF Question the CYK-Parser has to answear is: Does such a k exist?

  29. Answearing this question is easy: • Just try all possibilities • no problem since you are a computer ;-) • Range : from 1 to (l-1) • All the sets R s i,k and R s i+k , l-1 have already been computed at this point

  30. Transform our sample CF-grammar into Chomsky Normal Form • Overview • 1) eliminate ɛ-rules • 2) eliminate unit-rules • 3) remove non-productive non-terminals • 4) remove non –reachable non-terminals • 5) modify the rest until all grammar rules are of the form A  a , A  BC

  31. Our number grammar in CNF • Number(s)  0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 • Number(s)  Integer Digit • Number(s)  N1 Scale´ | Integer Fraction • N1  Integer Fraction • Integer 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 • Integer  Integer Digit • Fraction  T1 Integer • T1  . • Scale ´  N2 Integer • N2  T2 Sign • T2  e • Digit  0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 • Sign  + | -

  32. Building the recognition table • Input : Our example grammar in CNF input sentence: 32.5 e + 1

  33. Building the recognition table • 1) bottom-row : read directly from the grammar (rules of the form A a ) • 2) Check each RHS in the grammar

  34. Check each RHS of the grammar • Two Ways: Example: 2.5 e ( = s 2, 4) • 1) check each RHS e.g N1 Scale´ • 2) compute possible RH-Sides from the recognition table

  35. How this is done 1) N1 not in R s 2, 1 or R s 2, 2 N1 is a member of R s 2, 3 But Scale´ is not a member of R s 5, 1 2) R s 2, 4 is the set of Non- Terminals that have a RHS AB where either: A in R s 2, 1 and B in R s 3, 3 A in R s 2, 2 and B in R s 4, 2 A in R s 2, 3 and B in R s 5, 1 Possible combinations: N1 T2 or Number T2 In our grammar we do not have such a RHS, so nothing is added to R s 2, 4.

  36. Recognition table

  37. Recognition table (well-formed substring table)

  38. Computing R s i, l:follow the arrows V and W simultaneously A  BC , B a member of a set on the V arrow , C a member of a set on the W arrow

  39. Comparison • This process is much less complicated than the one we saw before • Why?

  40. Conclusion • This process is much less complicated • Reasons: 1) We do not have to repeat the process again and again until no new Non-Terminals are added to R s i,l (The substrings we are dealing with are really substrings and cannot be equal to the string we start with)

  41. Reasons cont. 2) We only have to find one place where the substring must be split into two A  B C Here !

More Related