Lecture 9

1. Lecture 9 sed

2. sed • sed is a stream-oriented editor • the input (file/std input) flows through the program sed and is directed the standard output • Used primarily for non interactive operations • sed [-n] –f script_file file or sed [-n] `command` file • sed executes the given command or script file that contain commands on each line of the input (file) • -n: turn off default printing

3. sed Commands • p: print line • -n prevents lines from being printed twice • d: delete line • s/old/new/: substitute old with new • s/old/new/g: substitute all occurrences of old with new • !: negates a command • Full list of commands can be found on page 129

4. sed Examples • sed p file.txt • sed –n p file.txt • sed d file.txt • sed \!d file.txt • p and d seem a bit worthless, don’t they? They purpose will become more clear when we discuss addresses.

5. sed: Substitution • The strongest feature of sed • Syntax is s/expression/string/flag • expression is a regular expression • string is a string • sed ‘s/|/:/’ data.txt • substitute the character ‘|’ with the character ‘:’ • sed ‘s/|/:/g’ data.txt

6. Some Useful Substitution Flags • g: global (replace all matches on the line). • p: print the line if a successful match • sed ‘s/old/new/g’ file.txt • sed ‘s/old/new/gp’ file.txt • sed –n ‘s/old/new/gp’ file.txt

7. Regular Expressions for sed • The usual suspects • ^, $, ., *, [ ], [^ ], \( \), \<, \> • A new operator • &: the string which matches the expression • can be used in the substitution string • s/hello/**&**/g replaces all occurrences of hello with **hello**

8. sed Addressing • So far, we have been applying sed commands to every line • makes p and d not very useful • With addressing, we can apply commands to some, but not all lines • sed can use • 0 addresses (all lines) • 1 address (a single line) • 2 addresses (a range of lines) • Address can be line numbers of context (defined by regular expressions)

9. Line Number Addressing Examples %sed –n ‘3,4p’ foo.txt Since sed prints each line anyway, if we only want lines 3 & 4 (instead of all lines with lines 3 & 4 duplicated) we use the –n %sed –n ‘$p’ foo.txt For each line, if that line is the last line, print %sed –n ‘3,$p’ foo.txt For each line, if that line is the third through last line, print

10. Context Addressing Examples • Use patterns/regular expressions rather than explicitly specifying line numbers %sed –n ‘/^From: /p’ $HOME/mbox • retrieve all the sender lines from the mailbox file, i.e., for each line, if that line starts with ‘From’, print it. Note that the / / mark the beginning and end of the pattern to match %ls –l | sed –n ‘/^.....w/p’ • For each line, if the sixth character is a W, print

11. Context Ranges • sed ‘/hello/,/there/d’ file.txt • delete all lines that occur between a line that matches hello and a line that matches there. The hello and there lines are also removed. • Multiple contexts are possible • two contexts specified by a single range

12. sed Addressing • Using a ! after the address means all lines which do not match the address • sed ‘1\!d’ test.txt

13. Example file northwest NW Charles Main 3.0 .98 3 34 western WE Sharon Gray 5.3 .97 5 23 southwest SW Lewis Dalsass 2.7 .8 2 18 southern SO Suan Chin 5.1 .95 4 15 southeast SE Patricia Heme 4.0 .7 4 17 eastern EA TB Savage 4.4 .84 5 20 northeast NE AM Main Jr. 5.1 .94 3 13 north NO Margot Webber 4.5 .89 5 9 central CT Ann Stephens 5.7 .94 5 13 sed ‘/north/p’ file sed –n ‘s/west/north/g’ file sed ‘3,$d’ file sed ‘s/\(Mar\)got/\1ianne/p’ file sed ‘s/west/north/g’ file sed ‘/west/,/east/s/$/**VACA**/’ file sed '/north/a\ hello,word' datafile

14. sed: Using files • Tedious to type in commands at the prompt, especially if commands are repetitive • Can put commands in a file and sed can use them • sed –f cmds.sed data.txt file with commands

15. sed scripts • Series of commands can be put in a file and use the ‘-f’ option. • Can also create an sed script: s/vi/emacs/g /[Ww]indows/d p

16. Another Example • sed script to remove all HTML tags from a file: s/<[^>]*>//g p

17. Reading Assignment • Chapter 5