1. Lesson 6-Using Utilities to Accomplish Complex Tasks

2. Overview • Creating and executing a script to list user information. • Listing directories and files separately. • Identifying changes made to files in a directory. • Creating a complex script.

3. Creating and Executing a Script to List User Information • Creating a script. • Executing a script.

4. Creating a Script • A series of commands can be executed repeatedly/automatically by placing them in a script file. • Placing commands in scripts helps avoid errors and save time. • The vi editor can be used to create a file. • The ":wq" commands are used to write to a file and return to the shell.

5. Executing a Script The commands in a file can be executed in two ways: • Instructions can be given to the current shell to read the file (source) and execute all the commands. • The script file can be made executable, and a child shell can be started to read the script file and execute the commands.

6. Executing a Script • The command used to make the shell executable is “chmod +x filename”. • By default, the output of the script is redirected to the screen. • The output of the script can be redirected to a file.

7. Listing Directories and Files Separately Selecting only directories / files: • The “ls –F | grep /” command is used to select only lines that contain directory names. • The “ls –F | grep / | column” command can be used to put the output into columns. • The “ls –F | grep –v / | column” command is used to select only files.

8. Listing Directories and Files Separately Selecting Directories Using Multiple Utilities

9. Listing Directories and Files Separately Interpretation by grep Using –v option

10. Listing Directories and Files Separately Adding comments to scripts: • The # (pound) sign is used to add comments to scripts. • Ensure that the # sign is placed at the beginning of each line. • The pound (#) sign cannot be used inside a long command line.

11. Identifying Changes Made to Files in a Directory Collecting data about files in a directory: • The “ls –l directory name” command can be used to provide information about the files in a particular directory. • The output of the code can also be redirected to another file by the command "ls -l directoryname > filename”. • The information includes the current status of each file, such as permissions, date of modification, owner, etc.

12. Identifying Changes Made to Files in a Directory Comparing file information: • The “comm” command is used to identify the common aspects between two files. • It requires two arguments – the names of the two the files that are to be compared. • The command gives three columns of output – lines unique in the first file, lines unique in the second file, and lines in common.

13. Identifying Changes Made to Files in a Directory Comparing file information: • The “diff” command is used to identify the differences between two files. • It requires two arguments – the names of the two files to be compared. • The command gives three columns of output – lines unique in the first file, lines unique in the second file, and lines common to the two files.

14. Creating a Complex Script • Determining the number of unique words in a file. • Removing punctuation. • Converting characters to lowercase. • Putting each word on a line. • Removing blank lines. • Sorting the lines.

15. Determining the Number of Unique Words in a File The “uniq” utility is used to: • Provide a list of unique words in a file. • The number of times that each word is used. • The number of unique words.

16. Determining the Number of Unique Words in a File • The uniq utility outputs both, unique lines as well as single copies of any lines that are duplicate and adjacent. • The utility deletes identical lines. • The uniq utility deletes only those lines that are identical and adjacent.

17. Determining the Number of Unique Words in a File The following things must be ensured for the uniq utility to work properly: • Punctuation and blank lines must be removed. • Differences in case for the same word must be reconciled. • The words must be one word to a line in a sorted order.

18. Removing Punctuation Shell and Utility Interpretation

19. Converting Characters to Lowercase Shell and tr Interpretation

20. Putting Each Word on a Line • To remove duplicate words using the uniq utility, the data must be modified to ensure that each word is on a line by itself. • The new line character “\n” is used to separate lines in a file. • The ASCII character code 012 can also be used to separate lines in a file.

21. Putting Each Word on a Line • The “man ascii” command is used to examine the ASCII characters and their associated codes. • TAB characters can also be replaced with new line characters.

22. Putting Each Word on a Line Using tr Utility to Replace Characters

23. Removing Blank Lines A View from the Shell

24. Sorting the Lines • The “sort” command can be used to sort the output such that all lines containing the same word are on adjacent lines. • The “uniq –c” command can be used to remove all duplicate lines. • The “sort –rn” command can be used to list the most frequently used words first.

25. Summary • A series of commands placed in a script file can be repeatedly executed, avoiding errors and saving time. • The ls utility provides a list of all the files and subdirectories in the current directory. • The # sign is used to add comments to script files. • The pipeline feature of UNIX is very useful and is central to manipulating data effectively with UNIX utilities.