460 likes | 561 Vues
This lecture covers advanced Bash programming tricks including the here document, trapping signals, and using commands like cut, tr, sed, and awk. Learn how to redirect text within the same file, handle signals efficiently, and manipulate text effectively using various filters and substitutions.
E N D
Introduction to Unix – CS 21 Lecture 12
Lecture Overview • A few more bash programming tricks • The here document • Trapping signals in bash • cut and tr • sed • awk
The Here Document • Redirects stdin to a specific set of text located inside the same file • << COMMAND << MARKER Data MARKER
Why Is This Useful? • Allows you to keep all relevant information in one file • Example: Database you want to search • Don’t need to clutter up your directory with unnecessary temporary files
Trapping Signals • Catching a signal is also called trapping a signal • You can tell bash programs what to do when they receive different signals • Analogy: When a postcard arrives, what do I do?
The ‘trap’ Command • Usage: trap ‘COMMAND’ Signals • Example: • trap ‘cat errorMsg’ 4 6 • In order to prevent you from running a program forever, signal number 9 cannot be trapped
Two Helper Filters • cut • Break individual lines apart • tr • Change characters into different characters
The ‘cut’ Command • More precise control over individual lines • Will cut out certain words from each individual line so they can be processed • Usage: cut [FLAGS] FILE
Flags • -d • Delimiter • -f • Field number • Example • cut –d: -f3 myFile
The ‘tr’ Command • Translate • Change on a one to one basis characters from one thing to another • Usage: tr ‘Set1’ ‘Set2’ • Example: tr ‘abc’ ‘ABC’ < myFile
Two More Powerful Tools • sed • Stream Editor • awk • Alfred Aho, Peter Weinberger, and Brian Kernighan
The ‘sed’ Command/Language • Filter • Like grep, sort, or uniq, it takes input and performs some operation on it to filter the output • Usage: sed ‘Address Command’ • Address specifies where to make changes • Command specifies what change to make • Example: • sed ‘4d’ textFile
Address Specification • Addresses could be line numbers or regular expressions • No address – each line • One address – only that line • Two comma separated addresses – All lines in between • ! – All other lines
Commands Available To sed • a\ • Append text • c\ • Replace text • i\ • Insert text before • d • Delete lines • s • Make substitutions
Substitution Example • Same syntax as vi
When Would You Want To Use sed? • sed works on streams, so it is perfect to be placed in the middle of a pipe in order to change the output from one format to another • Example: • If a program always prints out 4 lines of junk for every good line, sed can be used to weed out the junk
awk • Answers the question: • What do I do if I want to search for a pattern and actually use it? • Combination of grep and commands • Searches for some pattern or condition and performs some command on it • Complete programming language • Looks a lot like C syntactically • Variables are declared bash style
Pattern And Command • awk in its most basic form simply executes a command on all lines that match (or adhere to) a certain pattern • Usage: awk ‘Pattern { Command }’ FILE • Just like sed, if there is no pattern, then every line will be matched
Different Ways To Run Awk • awk ‘Pattern { Command }’ • awk –f awkFile inputFile • Since awk itself can be a complex language, you can store all the commands in a file and run it with the –f flag
Important awk Concepts • Record • Every line of an input file is a record • The current record can be referenced with $0 • Field • Every word in a record is called a field • Each field is numbered and can be referred to • $1 is the first record, $2 is the second, etc.
Special Predefined awk Variables • RS • The character that acts as a record separator • Default is the end of a line • FS • The character that acts as a field separator • Default is whitespace (space, tab, etc) • Can be redefined with the –F flag
Other awk Variables • NF = number of fields in the current record • NR = Total number of records seen so far • OFS = Output field separator • ORS = Output record separator
BEGIN And END Blocks • Two special patterns that can be matched • BEGIN • Commands are executed before any records are looked at • END • Commands are executed after all records are processed
Awk Patterns • /regular expression/ -> same as egrep • Relational expression • >, <, >=, <=, == • Pattern && pattern • Pattern || pattern • Pattern1 ? Pattern2 : pattern3 • If Pattern1 is True, then Pattern2, else pattern 3 • (pattern) • ! Pattern • Pattern1, pattern2
Awk Actions • Enclosed in { } • () Grouping • $ Field reference • ++ -- Increment, decrement • ^ Exponentiation • + - ! Plus, minus, not • * / % Multiplication, division, and modulus
Control Flow Statements • Inside of commands, you can have control flow • if • while • for
If Syntax if (condition) { Statements } else { Statements }
While Syntax while (Condition) { Statements }
For Syntax for (Declaration ; Condition ; Increment ) { Statements } for ( j=0; j < 5; j++) { print “hello world” }
When Would You Want To Use awk? • Whenever you want to search for some pattern and perform some action • Example: I want to go through and calculate the average score on the Midterm
Another Example • Adding 12 points to everyone’s midterm score
awk Versus bash $ arguments • Always enclose everything to awk in single quotes so they don’t get interpreted • $1 to awk means something completely different than $1 to bash • $1 in awk means first field • $1 in bash means first command line argument
Next Time • Looking at some of the string and mathematical awk functionality • find • Putting everything together • The complete bash programming necessities • Quiz 2 – Next Tuesday