Unix Commands

1. Unix Commands "Contrary to popular belief, Unix is user friendly. It's just very selective about who its friends are." --unknown Get your own quote – log on to mario, and then at the shell prompt, use the fortune command: $ fortune

2. First, some context… • What is an operating system? • Conceptually, it is software that abstracts the hardware • Provides process management, memory management, security • Interrupts, file system, networking • The OS is effectively a program (“kernel”) that runs in “privileged mode” • It can run all instructions • Kernel mediates access to computer's resources (CPU, RAM, I/O devices)

3. More on OSs • System calls • A program requests a service from the kernel • The API the OS exposes • UNIX has ~100 system calls • create process, context switch, dynamically load library, read, write, ioctl • sh% man 2 intro • System calls and library wrapper functions • Systems provide library (e.g., glibc) between normal programs and the OS • Call to the library function doesn't cause switch to kernel mode itself • Results in system call which transfers control to kernel • Ex: C library function fork()executes instructions that invoke fork system call

4. System Call vs. Library Call • In standard C, there are many library function calls • Some never leave user space • log, rand, strcpy, sscanf • Some always make system call(s) after some user-space code • fflush, fopen • Some sometimes make system call(s) after some user-space code • printf (write), fscanf (read), malloc (sbrk) • Some make system call immediately • syscall

5. Shell • shell: An interactive program that uses user input to manage the execution of other programs. • A command processor, typically runs in a terminal window. • User types commands, the shell runs the commands • Several different shell programs exist: • bash : the default shell program on most Linux/Unix systems • We will use bash • Other shells: Bourne, csh, tsch • Why should I learn to use a shell when GUIs exist?

6. Why use a shell? • Why should I learn to use a shell when GUIs exist? • faster • work remotely • programmable • customizable • repeatable

7. Popular Shells • sh (Bourne shell) • Emphasizes running programs and programmability • csh (C shell) • Lots of interactivity: history, job control, command/filename completion, aliases • C-like syntax • Traditionally not seen as a good choice for writing shell scripts • ksh (Korn shell) • Combines programmability and interaction (sh and csh’s interactivity) • bash (GNU shell) • Mostly ksh + much of csh • Default on Mac OS and cygwin • tcsh • Evolution of csh

8. Shell commands $ pwd /home/ecelrc/faculty/meberlein $ cd EE461L $ ls file1.txt file2.txt $ ls –l -rw-------. 1 meberlein faculty 0 Apr 18 19:31 file1.txt -rw-------. 1 meberlein faculty 0 Apr 18 19:36 file2.txt $ cd .. $ man ls $ exit

9. Unix file system

10. Directories and Navigation: Oh, the places you'll go... • absolute pathname: begins with root directory and specifies all branches to directory or file • relative pathname: starts from working directory

11. Directory commands • some commands (cd, exit) are part of the shell ("builtins") • others (ls, mkdir) are separate programs the shell runs

12. Shell commands • many accept arguments or parameters • example: cp (copy) accepts a source and destination file path • a program uses 3 streams of information: • stdin, stdout, stderr (standard in, out, error) • input: comes from user's keyboard • output: goes to console • errors can also be printed (by default, sent to console like output) • parameters vs. input • parameters: before Enter is pressed; sent in by shell • input: after Enter is pressed; sent in by user

13. Command-line arguments • most options are a - followed by a letter such as -c • some are longer words preceded by two - signs, such as --count • options can be combined: ls -l -a -r can be ls -lar • many programs accept a --help or -help option to give more information about that command (in addition to man pages) • or if you run the program with no arguments, it may print help info • for many commands that accept a file name argument, if you omit the parameter, it will read from standard input (your keyboard)

14. Shell commands • "man pages" : important way to learn new commands man ls man man

15. File commands • caution: the above commands do not prompt for confirmation • easy to overwrite/delete a file; this setting can be overridden (how?) • Exercise : Given several albums of .mp3 files all in one folder, move them into separate folders by artist. • Exercise : Modify a .java file to make it seem as though you finished writing it on Dec 28 at 4:56am.

16. Exercise Solutions • caution: the cp, rm, mv commands do not prompt for confirmation • easy to overwrite/delete a file; this setting can be overridden (how?) • Use “-i” with the command, “interactive” to prompt before overwrite • Exercise : Given several albums of .mp3 files all in one folder, move them into separate folders by artist. • mkdir U2 • mkdir PSY • mkdir JustinBieber • mv GangnamStyle.mp3 PSY/ • mv Pride.mp3 U2/ • Exercise : Modify a .java file to make it seem as though you finished writing it on Dec 28 at 4:56am. • touch –t 201712280456 Hello.java

17. Basic Emacs Commands • C- = control key M- = meta/alt key • read a file into Emacs: C-x C-f • save a file back to disk: C-x C-s • exit Emacspermanently: C-x C-c • search forward: C-s search backward: C-r • scroll to next screen: C-v scroll to previous screen: M-v • Undo: C-x u

18. Basic vi/vim Commands • :w Write the current file • :wq Write the current file and exit. • :q! Quit without writing • To change into insert mode: i or a • Use escape to exit • search forward /, repeat the search backwards: N • Basic movement: • h l k j character left, right; line up, down (also arrow keys) • b w word/token left, right • gee end of word/token left, right • 0 $ jump to first/last character on the line • x delete • u undo https://wiki.gentoo.org/wiki/Vim/Guide and http://tnerual.eriogerg.free.fr/vimqrc.pdf

19. File STUFF

20. Examine a File

21. Searching and sorting • grep : a powerful search tool... • Exercise : Given a text file students.txt, display the students arranged by the reverse alphabetical order of their last names.

22. Keyboard shortcuts ^KEY means hold Ctrl and press KEY

23. Streams • stdin, stdout, stderr • all default to the console • some commands expect an input stream – if you don't specify one, will read from the console • Example: grep yo

24. Output Redirection command > filename • run command and write its output to filename instead of to console • like an arrow going from the command to the file... • if the file exists? overwritten • >> appends rather than overwriting, if the file already exists • Example: ls -l > myfiles.txt • Example: java Foo >> Foo_output.txt • Example: cat > somefile.txt (writes console input to the file until you press ^D)

25. Input redirection command < filename • run command and read its input from filename instead of console • whenever the program prompts the user to enter input (such as reading from a Scanner in Java), it will instead read the input from a file • some commands don't use this - they accept a file name as an argument • Example: java Guess < input.txt • Exercise: Also change the output stream to write the results to file • again note that this affects user input, not parameters • useful with commands that can process standard input or files: • e.g. grep, more, head, tail, wc, sort, uniq, write

26. File Expansion • Brace expansion • mkdir -p hw1{old,new,test} • What does it do? • Replace pattern with a list of matching file names • ls ~/temp/[Dd][Ss][Cc]00*.jpg • What does it do?

27. Pattern Matching (in more detail) • * • Any string, including null string • ? • Any single character • [ ] • Any character from the set (e.g., [abc] or [a-c]) • [! ] or [^ ] • Any character not in the set (e.g., [!abc] or [^abc]) • Special case: “.” at the start of the filename • Hiddenfiles; treated differently • A couple of examples • mv mytaxes*19* very_old • mv mytaxes*200[0-9] old

28. Special Characters • Special characters can be difficult to use without eliciting their special meaning • How do you get around this? • Escape • \x takes the following character (E.g., x) literally • Single quotes • ‘xxx’ takes everything literally • Double quotes • “xxx” takes everything literally except for $, “, and \ if followed by a special character • Rules on what to escape or quote are arcane – when in doubt, just give it a go

29. Example • Directory contains three files: a.txt, a*.txt, and a?.txt • What do these do? • ls a?*.txt • ls a*.txt • ls a\*.txt • ls a\?\*.txt • ls “a?.txt” • ls “a?*.txt”

30. History • History is built into shells; uses the special ! character • !! • Last command • !n • Last command starting with n • !$ • Last argument of last command • fc command to run previous commands • fc –l ls > lastcommands • vi lastcommands • source lastcommands

31. Alias • Shorthand for commonly used commands • This is not the same as defining an environment variable • But these often go in your ~/.bashrc, too • Syntax: • alias ls=“ls -a” • To remove alias: • unaliasls

32. Combining Commands: Pipe • cmd1 | cmd2 • Run cmd1 and send its console output as input to cmd2 • i.e., send the output of cmd1 to cmd2 • Similar to the following: cmd1 > filename cmd2 < filename rm filename • Examples: history | grep man diff students.txt names.txt | less grep Mary names.txt | uniq • Exercise: Assume names.txt contains ECE student names, one per line, "LASTNAME, FIRSTNAME" format. We are interested in students whose first name begins with "M", such as "Eberlein, Mary". • How many such students are in the file?

33. Combining Commands in Sequence cmd1; cmd2 • Run cmd1 and then cmd2 afterward (they aren't linked) cmd1 && cmd2 • Run cmd1, and if it succeeds, run cmd2 afterwards • Doesn't run cmd2 if any error occurs during run of cmd1 Example: Make a directory called songs and move all .mp3 files into it. mkdir songs && mv *.mp3 songs

34. So do you get it?

35. The Soft Balls… • What does this do? • ls -l *.txt • How would you get help in using the command cp? • Write a command that lists all of the files in the current directory

36. Slightly Less Soft… • Create a new directory • Within your new directory, create and edit a new file called “my_new_file.txt” using vi • Without using vi, visualize the content of your new file • Rename your file to “my_old_file.txt”

37. Discover a new command… • How would you print the first 15 lines of all files in the current directory that end in “.txt”

38. Putting it together… • What does this do? • diff orig.txtcopy.txt | grep 1999 >> result.1999

39. Questions?