1 / 20

Data Structures for Grammars: Nullable and First Computation in SML

Explore a comprehensive lecture from Feb 12, 2007, covering data structures for grammars, computing Nullable and First in SML assignments. Discuss Project 1 deadlines and practical examples including code snippets. Dive into the fundamentals of operations on Grammars, with a focus on tables. Learn about updating and basic strategies for table management. Discover the process of initializing, recording changes, and applying rules for efficient data handling. The lecture emphasizes the importance of understanding Nullable and First computations through practical exercises and examples in SML.

jayden
Télécharger la présentation

Data Structures for Grammars: Nullable and First Computation in SML

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Lecture #9, Feb. 12, 2007 • A data structure for grammars • Computing Nullable and First in SML

  2. Assignments • Project 1 • Recall Project #1, the scanner is due Wednesday Feb. 14th • Midterm Exam. Recall the midterm is scheduled for 1 week from Today.

  3. fun inputc fname = (fn n => TextIO.inputN(TextIO.openIn fname,n):string); val lex = Mlex.makeLexer (inputc "test.english"); fun inputc h = (fn n => TextIO.inputN(h,n):string); val lex = let val h = TextIO.openIn “test.english” in Mlex.makeLexer (inputc h) end;

  4. Datatype for grammars val empty = "empty"; fun emptyP x = x=empty; datatype Grammar = Gram of (string list) * (* Non Terminals *) (string list) * (* Terminals *) (string * (string list)) list * (* Productions *) string; (* Start symbol *)

  5. Example Grammar val g1 = Gram(["Expr","Term","Factor","Expr'","Term'"] ,["ident","+","-","*","div","(",")","num"] ,[("Expr",["Term","Expr'"]) ,("Expr'",["+","Term","Expr'"]) ,("Expr'",["-","Term","Expr'"]) ,("Expr'",[]) ,("Term",["Factor","Term'"]) ,("Term'",["*","Factor","Term'"]) ,("Term'",["div","Factor","Term'"]) ,("Term'",[]) ,("Factor",["(","Expr",")"]) ,("Factor",["num"]) ,("Factor",["ident"]) ] ,"Expr");

  6. Operations on Grammars fun termP (Gram(nts,ts,ps,s)) x = List.exists (fn y => y=x) ts; fun nontermP (Gram(nts,ts,ps,s)) x = List.exists (fn y => y=x) nts; fun rhssFor (Gram(nts,ts,ps,start)) symbol = let fun test (lhs,rhs) = lhs=symbol fun rhs (l,r) = r in List.map rhs (List.filter test ps) end;

  7. Tables • Nullable, First, and Follow compute a table by incrementally adding stuff, until no new stuff can be added. • A table is simply a list of (string * a ref) pairs (* Create a table with all slots initialized by the function “initf” *) fun init symbols initf = let fun f x = (x,ref (initf x)) in List.map f symbols end;

  8. Updating • When we update a table we need to know if the update makes a change. Since we continue until no new changes are possible. (* Update the slot at "s" with the function "updatef" *) (* returns true if it makes a change, false otherwise *) fun update s updatef pairs = case List.find (fn (y,r) => s=y) pairs of SOME(_,r) => let val old = !r val new = updatef old in if new=old then false else (r := new; true) end | NONE => false

  9. Basic Strategy let val table = init … symbols val changed = ref true in while (!changed) do (changed := false ; changed := onePass symbols); table end • Initialize table • Record changes • Apply rules until no changes are made in a single pass.

  10. Code fun nullTable (gram as(Gram(nts,ts,ps,start))) = let val allsymbols = nts @ ts val table = init allsymbols (fn _ => false) val changed = ref true fun onePass [] = false | onePass (x::xs) = let val rhss = rhssFor gram x fun nullify b = List.exists (nullableRhs table) rhss val b1 = update x nullify table val b2 = onePass xs in b1 orelse b2 end in while (!changed) do (changed := false; changed := onePass allsymbols); table end

  11. and nullable table s = case List.find (fn (y,r) => y=s) table of NONE => false | SOME(_,r) => !r and nullableRhs table [] = true | nullableRhs table (x::xs) = (nullable table x) andalso (nullableRhs table xs);

  12. val NullT = nullTable g1; - NullT; val it = [("Expr",ref false), ("Term",ref false), ("Factor",ref false), ("Expr'",ref true), ("Term'",ref true), ("ident",ref false), ("+",ref false), ("-",ref false), ("*",ref false), ("div",ref false), ("(",ref false), (")",ref false), ("num",ref false)] : (string * bool ref) list

  13. Onepass for “first” (* returns true if it makes a change *) fun onePass [] = false | onePass (x::xs) = let val rhss = rhssFor gram x fun first old = let val listOflists = map (firstRhs gram table) rhss val new = List.concat listOflists in norm(old @ new) end val b1 = update x first table val b2 = onePass xs in b1 orelse b2 end

  14. First for a RHS and first table s = case List.find (fn (y,r) => y=s) table of NONE => [] | SOME(_,r) => !r and firstRhs gram table [] = [empty] | firstRhs gram table [x] = first table x | firstRhs gram table (x::xs) = let val temp = first table x in case List.find emptyP temp of NONE => temp | SOME _ => temp @ firstRhs gram table xs end;

  15. The main loop fun firstTable (gram as(Gram(nts,ts,ps,start))) = let val termTable = init ts (fn x => [x]) val nontermTable = init nts (fn x => []) val table = termTable @ nontermTable val changed = ref true fun onePass [] = false | onePass (x::xs) = let val rhss = rhssFor gram x fun first old = let val listOflists = map (firstRhs gram table) rhss val new = List.concat listOflists in norm(old @ new) end val b1 = update x first table val b2 = onePass xs in b1 orelse b2 end in while (!changed) do (changed := false; changed := onePass nts); table end

  16. val FirstT = firstTable g1; val it = [("ident",ref ["ident"]) ,("+",ref ["+"]) ,("-",ref ["-"]) ,("*",ref ["*"]) ,("div",ref ["div"]) ,("(",ref ["("]) ,(")",ref [")"]) ,("num",ref ["num"]) ,("Expr",ref ["(","ident","num"]) ,("Term",ref ["(","ident","num"]) ,("Factor",ref ["(","ident","num"]) ,("Expr'",ref ["+","-","empty"]) ,("Term'",ref ["*","div","empty"]) ] : (string * string list ref) list

  17. Possible exam questions • Define a grammar. • Prove that a particular grammar is ambiguous. • i.e. find a single string accepted by the grammar with more than 1 parse. • Give a regular expression that describes comments (or some other given lexeme). • Translate a regular expression into a finite state automata • where the FSA has <empty> transitions • where the FSA has no <empty> transitions • Be able to construct an abstract (inductive) data definition for binary trees (or some other data type)

  18. Possible exam questions (cont. 2) • Write a function over an abstract (inductive) data type by using patterns. • Produce the First, and Last sets from a regular expression. • Simulate a top-down parse of a string by hand for a particular grammar. • Remove left recursion from a grammar. • Layer an ambiguous grammar to produce non-ambigous grammar.

  19. Possible exam questions (cont. 3) • Write a recursive descent parser for a simple grammar. • Construct the First and Follow sets from a context free grammar • Given the first and follow sets, construct a predictive parsing table for a given grammar. Tell if such a grammar is ambiguous. • Simulate a bottom up (shift-reduce) parse by hand using a stack and an input stream, for a given string and a given grammar. • Disambiguate a shift reduce parser by using precedence and associativity information.

  20. Possible exam questions (cont. 4) • Write simple anonymous function in ML • Use the List library functions List.map, List.filter, List.exists, Lists.all, List.find to write simple functions.

More Related