TDC368 UNIX and Network Programming

1. TDC368UNIX and Network Programming Week 3-4: • UNIX File System • File Representation • File Handles • System Calls (APIs) • System Calls: dup dup2 pipe • Library functions: popen(); pclose(); Camelia Zlatea, PhD Email: czlatea@cs.depaul.edu

2. UNIX File System: files, catalogs (inodes; types,size, permissions). UNIX File I/O: unbuffered I/O functions (open ,close,seek, umask,read,write). Standard I/O buffering (fopen, fclose, fflush). File Sharing Communication between related processes via pipe (pipe, dup,dup2) Appendix: UNIX system calls: C library functions, C++ classes, POSIX API's. Outline

3. Hierarchical organization Device independence compatibility among files or catalogs, devices and IPC Types of files regular files (byte-stream, no format) directory files (groups of files mapped on user’s projects) special files (to access peripheral devices, /dev/...) pipe files (inter process communication mechanisms) UNIX File System

4. Regular Files Shell commands: cp, rm, mv, cat, ls System Calls: <fcntl.h>, <unistd.h> open, creat, read, write, close, link, unkink Directory Files Shell commands: mkdir, rmdir, mv System Calls: <dirent.h> Special Files mknod ( root privileges) Pipe Files mkfifo UNIX Type of Files

5. File Type (-,d, b,c,p) ls -l Access Permission for owner, group and others (r,w,x) File Owner ID and GID Number of hard links of a file File Size Last access, modify and change permission time INODE number File system ID File Attributes

6. Constant Attributes file type file inode number File System ID Changeable Attributes chmod (command and system call) chown, chgrp touch (command), utime (sys call) ln (command ), link(syscall) rm (command), unlink(syscall) Attributes query by: system calls stat or fstat File Attributes

7. all UNIX files have associated a set of owner permissions bits, that are used by the OS to determine the access classes of users: owner, group, other permissions: owner group other rwx rwx rwx 111 111 111 octal 777 111 101 101 octal 755 - default permissions = creation_mask XOR umask_value - creation mask (set at system initialization) is usually 0777 File Permissions

8. Command shell: umask new_mask_value (display/change umask value) System call: mode_t umask(mode_t mask); - to modify umask for all your processes edit your startup file , .login (.profile) and insert the line umask nnn (where nnn is the new umask value) - change permissions - command shell: chmod guorwx file File Permissions

9. System calls: <sys/types.h> <sys/stat.h> int chmod(const char *path, mode_t mode); int fchmod(int fid, mode_t mode); Example: Use system call stat and determine the inode information Name of the file is passed as program’s argument. File Permissions

10. #include <sys/types.h> #include <sys/stat.h> void main(int argc, char * argv[]) struct stat statbuf; if (stat(argv[1], &statbuf)==-1) { /* print error message */ exit(1); } else { if (statbuf.st_mode & S_IFDIR) { printf(“ %s is a directory \n”, argv[1]); printf(“user ID is %s \n”, stat_buf.st_uid); } } File Permissions

11. real user ID (UID) real group ID (GID) - information is obtained from the password file entry (/etc/passwd) group ID (GID) numbers should map to group names from /etc/group effective user ID (EUID) effective group ID (EGID) - usually UID identical with EUID File Owner Information

12. Example: command passwd (change password) need to have access to a system file (as a root) a non-privileged user is able to run a command temporarily as root if suid permission is set: rws r-x r-x root /etc/passwd File Owner Information

13. Command shell id is used to display owner information (UID, GUID, EUID, EGID) System Calls: <sys/types.h> <unistd.h> uid_t getuid(void); uid_t geteuid(void); gid_t getgid(void); gid_t getegid(void); File Owner Information

14. UNIX File System Organization: BOOT BLOCK SUPERBLOCK INODE TABLE DATA BLOCK Boot Block contains a program that loads UNIX kernel file from the file system transfer control to the start address of the kernel code if the system is already running and the disk is accessed by mount, boot block is ignored  SUPERBLOCK - contains all information that defines the current state of the file system - size of i-list Iinode list table) - size in blocks of the files system - number of free blocks available in the file system - the address of the free inodes list - number of total free blocks in the system - files system name and file system disk name - superblock is read when mounting the device  File Representation

15. INODE - identifies a UNIX file Inode Structure file attributes physical disk address where file date is located Inode Table (UNIX System V) an inode entry for each file in the file system Directory File - maps file name to inodes numbers File Representation

16. Inode Table inode structure File Information - size (bytes) - owner information (UID, GID, EUID, EGID) - relevant times - number of links - number of blocks - permissions Data Blocks - direct pointers to beginning of file blocks - indirect pointers - double indirect pointers - triple indirect pointers Data Block  ……………. Data Block File Representation

17. Directory Structure /root /bin /etc unix /dev /usr /tmp Absolute path name: /usr/project/bin/a.out Relative path name: ./a.out ../../a.out File name: NAME_MAX in <limits.h> Path name: PATH_MAX -’’- Hard Link - the path name of a file ln /usr/myproj/a.out /usr/project/bin/a.out

18. Directory Representation UNIX directory - a file containing a correspondence between filenames and inodes the inode does not contain file name data blocks of the directory file are directory entries a directory entry contains: inode number + file name command as rename/move (mv) affect the name filed from directory entry, only one physical copy of the file exists on disk (file may have several names on different directories or the same name) for a newly created directory (ex. mkdir dir_name) OS makes a new directory entry for it and assign an inode to it Directory Representation

19. Example of Directory file: Inode Number File Name 78 . 109 .. 115 sample 240 test.c 300 a.out 115 sample_data_l Accessing Files

20. Record-oriented files file name inode number Record data type is the structure dirent in SV/POSIX <dirent.h> direct in BSD <sys/dir.h> Directory Files

21. Opens a directory file, return a handle of type DIR* opendir Reads next record from directory file readdir Closes a directory file closedir Sets file pointer at the beginning rewinddir Random directory access telldir, seekdir Directory Functions

22. Hard Links: ln fileA fileB pathnames of a file (directory entries) NO new inode is created cannot link files across file systems cannot link directories (as reg.user) increase the hardlink counter per inode ln /dirA/name1 /dirB/name2 Effect: creates a hard link, both names point to the same inode structure) Symbolic Links: ln -s fileA fileB Symbolic link is a file containing the name of another file(directory) a new inode is created can link files across file systems can link directories (as reg.user) ln -s /dirB/name2 /dirA.name1 Effect: if there is an entry 12345 – name1 in /dirA creates new inode 56789 new entry 56789 – name2 in /dirB the block of date from inode 56789 contains “/dirA/name1” Hard vs. Symbolic Links

23. Header file: <sys/types.h> <fcntl.h> open - opens a file for data access Header file: <sys/types.h> <unistd.h> creat - creates regular file read - reads data from a file write - writes data to a file close - terminate the access to a file UNIX File System Calls (APIs)

24. Header files: <sys/types.h> <unistd.h> lseek - random access to a file link - creates a hard link to a file unlink - removes a hard link of a file chown - changes UID or GID of a file chmod - changes access perm. of a file Header files: <sys/types.h> <sys/stat.h> stat, fstat - queries attributes of s file UNIX File System Calls (APIs)

25. C stream pointers (FILE *) Conversions, header file <stdio.h> int fileno(FILE *stream_pointer) FILE *fdopen(inf fd, char *open_mode); C Library Func. UNIX System Call fopen open fread, fget, fgets, fscanf read fwrite,fput,fputs,fprintf write fseek, ftell, frewind lseek fclose close File Handles

26. FILE handle #include <stdio.h> FILE *fp; if (( fp = fopen(“test.dat”, “w”)== NULL) fprintf(stderr, “cannot open file\n”); else fprintf(fp, “This is a text”); fp “This is a text”); 3 User area Kernel area

27. Minimize # of write/read calls Types of Buffering Fully / Line / No Buffering Change Buffering Type #include <stdio.h> void setbuf(FILE *fp, char *buf) void setbuf(FILE *fp, char *buf, int mode, size_t size) mode: _IOFBF, _IOLBF, _IONBF size: default is BUFSIZ Flush a buffer int fflush(FILE *fp); Buffering

28. Parent Process opens file test.datbefore fork

29. Parent Process opens file test.datafter fork

30. Pipe • Think at a pipe as being a special file that can store a limited amount of data in a FIFO manner • Size: 512 K on most Unix systems (<limits.h> or <sys/param.h>) • Interaction rule: one process writes to the pipe (as it were a file) and other process reads from the pipe • Data are written at one end of the pipe and are read at the other end • System keeps track of the current location of last read/write

31. Process synchronization using pipes • If a writing process attempts to write to a full pipe the system blocks it until pipe is able to receive data • If a process attempts to read from an empty pipe the system blocks it until data is available • A process will block if it tries to read from a pipe that has been opened for reading but no process has opened it for writing • If a write is made to a pipe not open for reading SIGPIPE is generated and errno is set to ``broken pipe''

32. write() System Call • Data are written to a pipe using unbuffered I/O write • Include files: <unistd.h> • Calling pattern: • size t write (int fileds, void *buf, size t nbytes); • fileds is a file descriptor • Return values: • On success: #bytes written • On failure: ­1 • Set errno? Yes

33. Writing to a pipe • Each write request is appended to the end of the pipe • Write requests of P IPE BUF size or less are guaranteed not to be interleaved with other write requests to the same pipe • Write request may cause the process to block; the constants • O NONBLOCK, O NDELAY in !sys/fcntl.h? can be set by fcntl() to non­blocking strategy • When O NONBLOCK, O NDELAY are set and the request to write P IPE BUF size bytes or less fails the value returned by write is shown in the next slide Table ==>

34. Setting the constants that control piping strategy • The table shows what write returns in function of constants • Blocking constants for write() • If a write is made to a pipe not open for reading SIGPIPE is generated and errno is set to ``broken pipe''

35. read() system call • Data are read from a pipe using unbuffered I/O read • Include files: <sys/types.h>, <sys/uio.h>, <unistd.h> • Calling pattern: • size t read (int fileds, void *buf, size t nbyte); • fileds is a file descriptor; • Return values: • On success: #bytes read • On failure: ­1 • Set errno? Yes

36. Reading from a pipe • Read actions are initiated from the current position • If pipe is open for writing by another process but it is empty read returns as shown bellow • Blocking constants for read • If the pipe is not opened for writing by another process read returns 0 (as in case of O NDELAY set)

37. Types of pipes • unnamed pipes, used with related processes (example: parent/child or child/child) • named pipes, used with unrelated process

38. Unnamed pipes ; pipe() system call • An unnamed pipe is constructed with pipe() system call, • Include files <unistd.h> • Calling pattern int pipe ( int filedes[2] ); • Return values: • On success: 0 • On failure: ­1 • Set errno? Yes • Functionality: • pipe() operates on a full­duplex or half­duplex basis • Writing/reading from a pipe full­duplex half­duplex

39. Actions performed by pipe() • If successful, it returns two file descriptors, filedes[0], filedes[1] that reference two data streams • Historically pipes were non­duplex (i.e., unidirectional); if two­way communication was needed two pipes were opened, one for reading and the other for writing • Currently file descriptors returned by pipe() are full­duplex (i.e., bi­directional), both opened for reading and writing. • In a full duplex if the process writes to filedes[0] then filedes[1] is used for reading, and vice­versa • In a half­duplex, filedes[0] is used for reading and filedes[1] for writing • an attempt to use filedes[1] for reading or filedes[0] for writing produces an error

40. Example: a parent process sends to a child the first argument from its command line and child acknowledges the receipt /* Using a pipe to send data from parent to child */ #include <stdio.h> #include <unistd.h> #include <stdlib.h> #include <string.h> #include <errno.h> int main (int argc, char *argv[]) { int fdes[2]; static char message[BUFSIZ]; if (argc != 2) {fprintf(stderr,''Usage: %s message``n'', *argv); exit(1); } if (pipe(fdes) == ­1) { perror(''Pipe''); exit(2); } switch (fork()) { case ­1: { perror(''Fork''); exit(3); }

41. Example: a parent process sends to a child the first argument from its command line and child acknowledges the receipt /* Child process: closes fdes[1] and read fdes[0] to obtain the message */ case 0: { close(fdes[1]); if (read(fdes[0], message, BUFSIZ) != ­1) { printf(''Message received by child is: [%s]``n'', message); fflush(stdout); } else { perror(''Read''); exit(4); } break; }

42. Example: a parent process sends to a child the first argument from its command line and child acknowledges the receipt /* Parent closes fdes[0] and write message to fdes[1] */ default: { close(fdes[0]); if (write(fdes[1], argv[1], strlen(argv[1])) != ­1) { printf(''Message sent by parent is: [%s]``n'', argv[1]); fflush(stdout); } else { perror(''Write''); exit(5); } break; } } exit(0); }

43. Redirecting stdin and stdout cat cat • Examples: • cat > test.dat “2” “2” “0” “0” “1” “1” test.dat

44. Redirecting stdin and stdout; dup() system call • Create a pipe • Associate stdin and stdout with the created pipe using dup() system call • Include files <unistd.h> • Calling pattern • int dup ( int fdes ); • Return values: • On success: next available file descriptor • On failure: ­1 • Set errno? Yes or No

45. dup() functionality • dup() duplicates an original open file descriptor; • new descriptor references SFT (System File Table) entry for the next available non­negative file descriptor • new descriptor shares the same file pointer (offset), has the same access mode as original and remain open across exec. • since actions necessary for redirection are as in the code: • int fdes[2]; pipe(fdes); close(fileno(stdout));dup(fdes[1]); • there is a chance that descriptor returned by dup() is not the one closed;

46. dup2() system call • Using dup2() system call for file redirection • Include files <unistd.h> • Calling pattern int dup2 ( int fdes1, int fdes2 ); • Return values: • On success: next available file descriptor >=0 • On failure: ­1 • Set errno? Yes • dup2() functionality: • dup2() closes the file and duplicates the file descriptor atomically • therefore dup2() avoid the limitation of dup() • both dup() and dup2() can be implemented with fcntl()

47. Redirecting stdin and stdout • Example: int fd; mode_t fd_mode = S_IRUSR | S_IWUSR | S_IRGRP | S_IROTH; fd = open(“test.dat”, O_WRONLY | O_CREAT, fd_mode); dup2(fd, STDOUT_FILENO); ….. close(fd); Process FDT after close Process FDT after dup2 Process FDT after open

48. Example: implementing ''last | sort '' command default: { dup2(fdes[0],fileno(stdin)); close(fdes[0]); close(fdes[1]); execlp(''sort'', ''sort'', 0); exit(4); } } } /* A home grown last | sort command */ #include <stdio.h> #include <unistd.h> #include <stdlib.h> #include <errno.h> void main (void) { int fdes[2]; if (pipe(fdes) == ­1) { perror(''Pipe''); exit(1); } switch (fork()) { case ­1: { perror(''Fork''); exit(2); } case 0: { dup2(fdes[1], fileno(stdout)); close(fdes[0]); close(fdes[1]); execlp(''last'', ''last'', 0); exit(2); }

49. Per Process FDT and SFT before dup2()

50. Per Process FDT and SFT after dup2()