Understanding sed: Stream-Oriented Text Editing with Substitutions and Addressing

1. Lecture 7.1 Sed

2. sed • sed is a stream-oriented editor • the input (file/std input) flows through the program sed and is directed the standard output • Used primarily for non interactive operations • sed [-n] –f script_file file or sed [-n] command file • sed executes the given command or script file that contain commands on each line of the input (file) • -n: turn off default printing

3. sed Commands • p: print line • -nprevents lines from being printed twice • d: delete line • s: substitution • s/old/new/: substitute old with new • s/old/new/g: substitute all occurrences of old with new • !: negates a command

4. sed Examples • sed p file.txt (print each line twice) • sed –n p file.txt • sed d file.txt (print nothing) • sed \!d file.txt (print each line) • p and d seem a bit worthless, don’t they? They purpose will become more clear when we discuss addresses.

5. sed: Substitution • The strongest feature of sed • Syntax is s/expression/string/flag • expression is a regular expression • string is a string • sed ‘s/:/|/’ /etc/passwd • substitute the character ‘|’ with the character ‘:’ • sed ‘s/:/|/g’ /etc/passwd • apply to all matches in the line

6. Some Useful Substitution Flags • g: global (replace all matches on the line). • p: print the line if a successful match • sed –n ‘s/old/new/g’ file.txt • sed ‘s/old/new/gp’ file.txt • sed –n ‘s/old/new/gp’ file.txt

7. Regular Expressions for sed • The usual R.E. operators • ^, $, ., *, [ ], [^ ], \( \), \<, \>, • A new operator • &: the string which matches the expression • can be used in the substitution string • s/hello/**&**/g replaces all occurrences of hello with **hello**

8. sed Addressing • So far, we have been applying sed commands to every line • makes p and d not very useful • With addressing, we can apply commands to some, but not all lines • sed can use • 0 addresses (all lines) • 1 address (a single line) • 2 addresses (a range of lines) • Address can be line numbers of context (defined by regular expressions)

9. Line Number Addressing Examples %sed –n ‘3,4p’ foo.txt Since sed prints each line anyway, if we only want lines 3 & 4 (instead of all lines with lines 3 & 4 duplicated) we use the –n %sed –n ‘$p’ foo.txt # $=last line For each line, if that line is the last line, print %sed –n ‘3,$p’ foo.txt For each line, if that line is the third through last line, print

10. Context Addressing Examples • Use patterns/regular expressions rather than explicitly specifying line numbers %sed –n ‘/^From: /p’ $HOME/mbox • retrieve all the sender lines from the mailbox file, i.e., for each line, if that line starts with ‘From’, print it. Note that the / / mark the beginning and end of the pattern to match %ls –l | sed –n ‘/^.....w/p’ • For each line, if the sixth character is a W, print

11. Context Ranges • sed ‘/hello/,/there/d’ file.txt • delete all lines that occur between a line that matches hello and a line that matches there. The hello and there lines are also removed. • Multiple contexts are possible • sed ‘/hello/,/there/d; /if/,/end/d’ file.txt

12. Example file northwest NW Charles Main 3.0 .98 3 34 western WE Sharon Gray 5.3 .97 5 23 southwest SW Lewis Dalsass 2.7 .8 2 18 southern SO Suan Chin 5.1 .95 4 15 southeast SE Patricia Heme 4.0 .7 4 17 eastern EA TB Savage 4.4 .84 5 20 northeast NE AM Main Jr. 5.1 .94 3 13 north NO Margot Webber 4.5 .89 5 9 central CT Ann Stephens 5.7 .94 5 13 sed ‘/north/p’ file sed -n ‘s/west/north/g’ file (no print out, need a p) sed ‘3,$d’ file sed ‘s/\(Mar\)got/\1ianne/p’ file sed ‘s/west/north/g’ file sed ‘/west/,/east/s/$/**VACA**/’ file

13. sed: Using files • Tedious to type in commands at the prompt, especially if commands are repetitive • Can put commands in a file and sed can use them • sed –f cmds.sed data.txt file with commands

14. sed scripts • Series of commands can be put in a file and use the ‘-f’ option. • Can also create an sed script: s/vi/emacs/g /[Ww]indows/d p

15. Another Example • sed script to remove all HTML tags from a file: s/<[^>]*>//g p

16. Announce Lab 3 http://www.cse.ohio-state.edu/~lido/teaching/sp2013cse4251/lab3.html