Download
unix filters n.
Skip this Video
Loading SlideShow in 5 Seconds..
Unix Filters PowerPoint Presentation
Download Presentation
Unix Filters

Unix Filters

493 Vues Download Presentation
Télécharger la présentation

Unix Filters

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. Unix Filters Commands with a dual purpose

  2. Objectives • Reinforce Directory and File Commands • Review Redirection • Introduce Filters: • head • tail • more • less • sort • uniq • wc • tee • cut • Introduce Pipes

  3. Filters • Filter commands – Unix commands that serve dual purposes: • standalone • used with other commands and pipes

  4. Definition of a Filter • A filter is a program that: • receives input from the standard output (stdout) of a Unix command, typically as text; • transforms that output, according to a parameterized set of user guidelines; and, finally, • writes that output to the standard output (stdout) or possibly to another filter.

  5. Filter Commands • Filter commands include: • head • tail • more • less • sort • uniq • wc • tee • cut • grep* • awk* * to be covered later

  6. Filters can be used with or without Pipes • Examples: • with: ls | wc –l counts the number of files in the directory • without wc –l abc.txt Counts the number of lines in the file abc.txt

  7. Pipes and Filters pipe Unix Command filter output • Example: ls | wc –l • This particular command: • gets a list of the files in the directory • sends this list through the pipe • the filter counts the lines of the data received • The result is equivalent to: The number of files in the directory

  8. What do pipes do? Using the example: ls | wc –l to do this without a pipe would require something like this: ls >tempfile # list contents of dir, redirect to tempfile wc –l tempfile # count the lines in tempfile (i.e. tell me the number of files) rmtempfile # delete the tempfile

  9. Using cat with pipes • Avoid using cat with pipes unless the cat command will transform the data through the use of options! • Good: cat –n abc | more # the cat adds line numbers to the listing of file abc • Bad: catabc | more # the cat does not alter the listing at all. In this case simply use more as a command as in: moreabc

  10. Redirection Examples cat <abc >def3 cat >abc cat >>abc cat abc1 cat <abc2 1.This assumes the file abc does exist. 2.This example is merely a demonstration of redirecting from a file. It is the equivalent of: cat abc 3. This is equivalent to: cp abc def

  11. Redirection • Redirection deals with changing the source or destination of the standard input and output. • Standard (I/0): • input : the keyboard • redirect is the less than (<) symbol • output: the computer screen • redirect is: • greater than symbol (>) to create a new output file (if file exists writes over it) • two greater than symbols (>>) append to existing file or if not an existing file create a new one

  12. Pipes versus Redirection • Pipes (|) send output from a Unix command to another Unix command (filter) • Redirection: • output (>, >>) send output of Unix command to a device (usually a file) other than the standard output (the screen) • input (<) provide input to a Unix command from a device (usually a file) other than the standard input (the keyboard)

  13. Pipes versus Redirection std out std in no redirection Unix command < > redirection (<,>) device to/from a Unix command disk disk output input Unix command Unix command pipe (|) Unix Command to Unix Command

  14. head • head outputs the first part of files • Format: head [options] filename • Selected options: • -cn : print the first n characters of the file • -n : print the first n lines the default is 10

  15. head Examples • Command: head –c5 abc # displays first 5 characters (includes whitespace) headabc # displays first 10 lines of abc • Pipe ls | head -15 #displays first 15 lines of file listing

  16. tail • tail outputs the last part of files • Format: tail [options] filename • Selected options: • -cn : print the last n characters of the file • -n : print the last n lines the default is 10

  17. tail Examples • Command: tail –c5 abc # displays last 5 characters (includes whitespace) tail abc # displays last 10 lines of abc • Pipe ls | tail -15 #displays last 15 lines of file listing headabc | tail -5 # prints lines 6 through 10 of file abc

  18. more • more displays files on a page-by-page basis • Format: more [options] filename • Options (selected): • -n : number of lines to show at a time • -d :prompt to continue • +/pattern : begin at pattern • Commands to use while in more: • space : advance a page at a time • return : advance a line at a time • q : quit more

  19. more Examples • Command: moreabc # lists file abc one page/line at a time • Pipe: ls –l | more # lists directory contents one page/line at a time

  20. less • less displays files on a page-by-page basis • Format: less [options] filename • Options (selected): • -E : quit at EOF (does not require the q) • -N : show line numbers • -ppattern : begin at pattern • Commands to use while in less: • h : help (try it, you’ll like it) • space : advance a page at a time • return : advance a line at a time • q : quit less

  21. less Examples • Command: less abc # lists file abc one page/line at a time • Pipe: ls –l | less # lists directory contents one page/line at a time

  22. sort • sort sorts lines of text [files] • usually (but not always) used with redirection • Format: sort [options] • Options (selected): • -b : ignore leading blanks • -f : ignore case • -r : sort descending rather than ascending • -n : sort numerical rather than by string • -u : unique, eliminate duplicate lines • -tchar : use char as field separater (instead of whitespace)

  23. sort Anomaly • Caution: If multiple, adjacent delimiters are found, only the first delimiter is treated as a delimiter! This includes the default delimiter (whitespace) • For example:

  24. sort • When using fields to sort, you may also use characters within a field. • Examples: • sort –t: +4 -5 /etc/passwd (uses colon as field delimiter and does the 5th field only) • sort –k 1.2 shuffled (sorts on 2nd character (and beyond) in first field) • sort –nu myfile (sorts starting with first field, values in numerical order, removing duplicates)

  25. Sort Fields • Examples: • sort –k 2 filename # sorts on last name (Smith, Ye, etc…) • sort –t, -k 2 filename # sorts on first name, month, and day (field2 of comma delimited fields) • sort –t: -k 1.8 filename # sorts on 2nd letter of last name (this assumes that the 2nd letter of the last name is in the 8th column of the first field. There is no colon (:) delimiter in the data making the entire line the 1st field

  26. sort • When sorting on multiple fields, you may also use the –k pos1[,pos2] options. • If you wish to format the search, place the format value adjacent to the field number. • Examples:

  27. uniq • uniq removes duplicate adjacent lines from a file • To ensure that all of the data is unique you should sort the file first • Format: uniq [options] filename • Options (selected): • -c : count the instances • -d : print only the duplicates (once) • -u : print only the unique lines • -n : ignore first n fields

  28. uniq Examples • Command: uniq –u abc # only prints out the unique lines in file abc assuming duplicates are next to each other uniq –c abc # only prints out the count of unique lines assuming duplicates are next to each other • Pipe: sortabc | uniq –c # put abc in order and tell me how many lines are unique

  29. wc • wcprints the number of newlines, words, and characters (bytes) in files • Format: wc [options] filename • Options: • -c : print the number of characters in the file • -w : print the number of words in a file • -l : print the number of lines in a file • If no options given, wc prints newlines, words, and bytes in files in that order

  30. cut • cut removes columns or fields from a line • Format: cut [options] filename • Options: • -clist* : the list of columns to cut • -flist * : the list of fields to cut • -dchar : the delimiter for fields (whitespace is the default) • -s : suppress lines without delimiters *list may be specified using integers separated by commas, ranges are sparated by hyphens (-) Example: 1,5-7 means units 1,5,6,7

  31. cut • Examples: cut -d: -f5 /etc/passwd (print only the 5th field, delimited by colon(:), of the /etc/passwd file) cut –d” “ –f2 shuffled (print only the 2nd field, delimited by a space, from file shuffled) cut –c4-8 shuffled (print only the columns 4 through 8 of each line in shuffled)

  32. tees • A filter that permits redirection to a file and standard output (screen) • Requires a pipe • Format: cmd| tee outfile where: • cmd is any Unix command that produces standard output • outfile is the file receiving the output data

  33. tee Examples • Filter only: ls –al ~ | tee dirlist # displays a long listing of all the files in your home directory and write the same to the dirlist file. ls –al ~ | tee dirlist | sort –r # displays a long listing of all the files in your home directory in reverse order, but saves the listing in order to the file dirlist.

  34. Pipes and tee tee disk Unix Command pipe screen