1 / 15

Lex and Yacc

Lex and Yacc. COP - 3402. General Compiler Infra-structure. Syntactic Structure. Program source (stream of characters). Tokens. Parser. Semantic Routines. Scanner (tokenizer). Lex. Yacc. IR: Intermediate Representation (1). Analysis/ Transformations/ optimizations.

clare
Télécharger la présentation

Lex and Yacc

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Lex and Yacc COP - 3402

  2. General Compiler Infra-structure Syntactic Structure Program source (stream of characters) Tokens Parser Semantic Routines Scanner (tokenizer) Lex Yacc IR: Intermediate Representation (1) Analysis/ Transformations/ optimizations Symbol and Attribute Tables IR: Intermediate Representation (2) Code Generator Assembly code

  3. Lex & Yacc • Lex • generates C code for the lexical analyzer (scanner) • Token patterns specified by regular expressions • Yacc • generates C code for a LR(1) syntax analyzer (parser) • BNF rules for the grammar

  4. Lex • lex is a program (generator) that generates lexical analyzers, (widely used on Unix). • It is mostly used with Yacc parser generator. • Written by Eric Schmidt and Mike Lesk. • It reads the input stream (specifying the lexical analyzer ) and outputs source code implementing the lexical analyzer in the C programming language. • Lex will read patterns (regular expressions); then produces C code for a lexical analyzer that scans for identifiers.

  5. Lex predefined variables and functions

  6. Lex – Pattern Matching Primitives

  7. Lex – Pattern Matching Examples

  8. Example: Simple Calculator • Computes the basic arithmetic operations • Allows declaration of variables • Enough to illustrate the basic structure of Lex and Yacc programs

  9. Lex program structure … definitions … %% … rules … %% … subroutines … %{ #include <stdio.h> #include "y.tab.h" int c; extern int yylval; %} %% " " ; [a-z] { c = yytext[0]; yylval = c - 'a'; return(LETTER); } [0-9]* { yylval = atoi(yytext); return(NUMBER); } [^a-z0-9\b] { c = yytext[0]; return(c); }

  10. Pattern Matching and Action Match a character in the a-z range Buffer [a-z] { c = yytext[0]; yylval = c - 'a'; return(LETTER); } [0-9]* { yylval = atoi(yytext); return(NUMBER); } Place the offset c – ‘a’ In the stack Match a positive integer (sequence of 0-9 digits) Place the integer value In the stack

  11. Yacc • Grammars described by rules using a variant of the Backus Naur Form (BNF) • Context-free grammars • LALR(1) parse table is generated automatically based on the rules • Actions are added to the rules and executed after each reduction

  12. Yacc Program Structure %{ #include <stdio.h> int regs[26]; int base; %} %token NUMBER LETTER %left '+' '-‘ %left '*' '/‘ %% list: | list stat '\n' |list error '\n' {yyerrok;} ; stat: expr {printf("%d\n",$1);} | LETTER '=' expr {regs[$1] = $3;}; expr: '(' expr ')'   {$$ = $2;} | expr '+' expr {$$ = $1 + $3;} | LETTER {$$ = regs[$1];} %% main(){return(yyparse());} yyerror(CHAR *s){fprintf(stderr, "%s\n",s);} yywrap(){ return(1);} … definitions … %% … rules … %% … subroutines …

  13. Rule Reduction and Action Action Grammar rule stat: expr {printf("%d\n",$1);} | LETTER '=' expr {regs[$1] = $3;}; expr: expr '+' expr {$$ = $1 + $3;} | LETTER {$$ = regs[$1];} “or” operator: For multiple RHS

  14. Further reading… • “A Compact Guide to Lex & Yacc”, Thomas Niemann (recommended); • “Lex & Yacc”, Doug Brown (O’Reily); • Lots of resources on the web • Check our website for some suggestions

  15. Conclusions • Yacc and Lex are very helpful for building the compiler front-end • A lot of time is saved when compared to hand-implementation of parser and scanner • They both work as a mixture of “rules” and “C code” • C code is generated and is merged with the rest of the compiler code

More Related