Files and Directories

1. Files and Directories

2. File Types in Unix • regular file - can be text, binary, can be executable • directory file - "folder type" file • FIFO file - special pipe device file, allows unrelated processes to communiate • block device file - represents physical device that transmit data a block at a time • character device file - represents physical device that doesn't necessarily transmit data a block at a time • symbolic link file - contains a pathname that references another file (some versions of UNIX)

3. creating and removing files • directory created using mkdir, removed using rmdir mkdir /usr/tmp/junkdir rmdir /usr/tmp/junkdir • device files created using mknod • (need to be superuser) mknod/dev/cdsk c 115 5 mknod/dev/bdsk b 287 101

4. Creating and removing files • FIFO created using mkfifo mkfifo /usr/tmp/fifo.mine • to make a link, use ln: ln -s /usr/jose/original /usr/mary/slink • then, if you type more /usr/mary/slink • you get the contents of /usr/jose/original ... the link is followed (like a pointer)

5. File attributes • file type • access permission - for owner, group, other • hard link count • uid - user id of the file owner (linked to user via password file) • gid - group id of the file owner • file size - in bytes • last access time • last modify time • last change time – time file permission, uid, gid, or hard link count changed • inode number - syteminode number of the file • file system id • major and minor device numbers

6. File attributes • You can see many of the file attributes by typing: ls -l -rw-r--r--. 1 eileen users 199 Apr 2 04:41 arithmetic.c -rw-r--r--. 1 eileen users 142 Apr 2 04:41 ascii.c -rw-r--r--. 1 eileen users 205 Apr 2 04:41 basic_types.c -rw-r--r--. 1 eileen users 91 Apr 2 04:41 bit.c -rw-r--r--. 1 eileen users 253 Apr 2 04:41 bitmask.c -rw-r--r--. 1 eileen users 155 Apr 2 04:41 blocks.c -rw-r--r--. 1 eileen users 6021 Apr 1 08:41 buggy.c -rw-r--r--. 1 eileen users 6178 Apr 1 08:33 commented.c

7. File attributes • used by the kernel in managing files • uidand gidare compared against those of a process attempting to access the file to determine which access permissions apply • The make utility looks at the last modification time • Not all fields make sense for every file: • size doesn't apply to device files • device number doesn't apply to regular files

8. Some file attributes can’t be changed • inode number • file type • file system id • major and minor device number

9. Changing file attributes

10. i-nodes and directory structure • kernel does NOT use file names to identify files • file names are not even one of the attributes that the kernel maintains • the kernel’s file system has an i-node table to keeps track of all the files • i-node table consists of i-node records that contains the file's attributes, the physical address where the file is stored • i-node number identifies the record within the table • unique within a file system

11. What’s in an i-node record? • stat, lstat, or fstatwill give you most of it stat Apr_02_2014.html File: `Apr_02_2014.html' Size: 693 Blocks: 8 IO Block: 4096 regular file Device: fd0ch/64780d Inode: 32637256 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 1012/eileen) Gid: (100/users) Access: 2014-04-02 04:42:09.028711229 -0400 Modify: 2014-04-02 04:42:09.028711229 -0400 Change: 2014-04-02 04:42:09.029711272 -0400

12. i-node records • calls to fstat, lstat return a struct stat • What’s a struct? • a complex data type declaration that defines a physically grouped list of variables placed under one name in a block of memory • allows the different variables to be accessed via a single pointer or the struct declared name that returns the same address

13. struct stat struct stat{ dev_tst_dev; /*file system ID */ ino_tst_ino; /*file inode number */ mode_tst_mode; /*file type &access flags*/ nlink_tst_nlink; /*hard link count */ uid_tst_uid; /*file user ID */ gid_tst_gid; /*file group ID */ dev_tst_rdev; /*major & minor device nums*/ off_tst_size; /*file size in bytes*/ time_tst_atime; /*last access time */ time_tst_mtime; /*last modification time*/ time_tst_ctime; /*last status change time*/ };

14. Directory structure • directories are files • contain the names and i-nodes of files in that directory

15. Links: hard and symbolic • "Hard links” • directory entries that associate a file name with an inode number. • "Soft links” (symbolic links) • files that contain the name (absolute or relative path name) of another file. • followed by most commands and have the result of accessing the file named inside the soft link file

16. process-level and kernel level data structures involved in managing file access

17. UNIX file access primitives • open - open for reading, writing, or create empty file • creat- create an empty file • close - close a file • read - get info from file • write - put info in file • lseek- move to specific byte in file • unlink - remove a file • remove - remove a file • fcntl- control attributes associated w/ file

18. open, read, close #include <fcntl.h> #include <unistd.h> main() { intfd; ssize_tnread; char buf[1024]; /* open file "data" for reading */ fd = open("data", O_RDONLY); /* read in the data */ nread = read(fd, buf, 1024); /* close the file */ close(fd); }

19. open for reading … /* open file "data" for reading */ fd = open("data", O_RDONLY); /* if fd is negative -- an error (-1) if fd is non-negative, it is a file descriptor that identifies the file and is used to further access it */

20. “man open” #include <sys/types.h> #include <sys/stat.h> #include <fcntl.h> intopen(const char *path, intoflag, /* mode_t mode */...); path -- pathname, can be relative or absolute oflag -- mode (defined in fcntl.h) O_RDONLY - read only O_WRONLY - write only O_RDWR - read write return value: -1 indicates error other value - index into file descriptor table (size of table determines number of open files possible) mode -- optional, used only with O_CREAT (more on this later)

21. read in the data … nread = read(fd, buf, 1024); /* try to read 1024 bytes from the file referred to by fd into the memory named "buf" */ …

22. “man read” SYNOPSIS #include < unistd.h > ssize_t read(intfildes, void *buf, size_tnbyte); try to read nbyte bytes into buf from file referred to by filedes return -1 or number of bytes read #define BUFSIZE 1024 char buf[BUFSIZE]; intbytes_read; intfd = open("somefile", O_RDONLY); bytes_read = read(fd, buf, BUFSIZE);

23. close the file … /* close the file */ close(fd); /* releases that fd for future use*/ /* program termination would have same effect */ …

24. “man close” SYNOPSIS #include <unistd.h> int close(intfd); DESCRIPTION close() closes a file descriptor, so that it no longer refers to any file and may be reused. … If fd is the last file descriptor referring to the underlying open file description the resources associated with the open file description are freed …

25. open for read & write ##include < stdlib.h > #include < fcntl.h> char *workfile="junk"; main(){ intfiledes; if ((filedes = open(workfile, O_RDWR)) == -1){ printf("Couldn't open %s\n", workfile); exit(1); /* abnormal (error) exit */ } /* read/write in loop until EOF */ exit(0); /* normal exit */ }

26. open for write #include <stdlib.h> #include <fcntl.h> #define PERMS 0644 char *filename="newfile"; main(){ intfiledes; // note: if filename already exists it will open if ((filedes = open(filename, O_WRONLY|O_CREAT, PERMS)) == -1){ printf("Couldn't open %s\n", filename); exit(1); /* abnormal (error) exit */ } /* read/write in loop until EOF */ exit(0); /* normal exit */ }

27. open … refinements If instead, we had written: if ((filedes = open(filename, O_WRONLY|O_CREAT|O_EXCL, PERMS)) == -1) then would fail if file already exists. if we had written: if ((filedes = open(filename, O_WRONLY|O_CREAT|O_TRUNC, PERMS)) == -1) if file already exists, would succeed, but would truncate it to size 0.

28. creat – old way to open a file creat- create a new file or rewrite an existing one SYNOPSIS #include < sys/types.h > #include < sys/stat.h > #include < fcntl.h> intcreat(const char *path, mode_t mode); returns fd or -1 always truncates an existing file always opens for writing only fd= creat("/users/students/you/testfile", 0644); is equivalent to: fd= open("/users/students/you/testfile", O_WRONLY|O_CREAT|O_TRUNC, 0644);

29. File pointer (R/W pointer) #define BUFSIZE 1024 char buf1[BUFSIZE]; char buf2[BUFSIZE]; intbytes_read; intfd = open("somefile", O_RDONLY); bytes_read = read(fd, buf1, BUFSIZE); bytes_read = read(fd, buf2, BUFSIZE); • each read gets the nextchunk of bytes • system keeps track of location in file with file pointer, a.k.a. the read/write pointer • maintains the number of the next byte to be read or written through that file descriptor

30. File pointer • sequential access to files – • just keep reading/writing, system updates for you • random access to files – • programmer explicitly changes the position of the read/write pointer. • How? with lseek • How do we know when we're finished reading??? • read returns a zero • Note: read attempt before last may succeed in reading fewer than requested bytes ...

31. Example #include <unistd.h> #include <stdlib.h> #include <fcntl.h> #include <stdio.h> #define BUFSIZE 512 main(){ char buffer[BUFSIZE]; intfiledes; ssize_tnread; long total = 0; /* open the test file */ if ((filedes = open("test.txt", O_RDONLY)) == -1){ printf("Couldn't open test.txt \n"); exit(1); } …

32. Example, continued /* read in loop until EOF */ while ( (nread = read(filedes, buffer, BUFSIZE)) > 0) { printf("Read %d bytes this time \n", nread); total += nread; } printf("Read %d bytes in total \n", total); exit(0); /* normal exit */ } For maximum efficiency, read in chunks same as system block size -- check BUFSIZ in /usr/include/stdio.h to find out.

33. write SYNOPSIS #include < unistd.h > ssize_t write(intfildes, const void *buf, size_tnbyte); ... DESCRIPTION The write() function attempts to write nbyte bytes from the buffer pointed to by buf to the file associated with the open file descriptor, fildes. returns -1 (error) or number of bytes written.

34. write intfd; ssize_t w1, w2; char buf1[512], buf2[1024]l ... fill buf1 and buf2 if((fd=open("somefile", O_WRONLY | O_CREAT | O_EXCL, 0644))== -1) return (-1) w1 = write(fd, buf1, 512); w2 = write(fd, buf2, 1024); • will write 1536 bytes to somefile, overwriting anything there already • to avoid over-writing: fd = open("somefile", O_WRONLY | O_APPEND);

35. Example • See copyfile.c • Then alter copyfile to accept BUFSIZE as a parameter as an input paramater and time it time copyfilein.tstout.tst 1 time copyfilein.tstout.tst64 time copyfilein.tstout.tst128 time copyfilein.tstout.tst256 …

36. lseek, random access SYNOPSIS #include < sys/types.h > #include < unistd.h> off_tlseek(intfildes, off_t offset, int whence); DESCRIPTION The lseek() function sets the file pointer associated with the open file descriptor specified by fildes as follows: • If whence is SEEK_SET, the pointer is set to offset bytes. • If whence is SEEK_CUR, the pointer is set to its current location plus offset. • If whence is SEEK_END, the pointer is set to the size of the file plus offset. • Returns resulting offset or error

37. lseek example Sample use: fd = open (filename, O_RDWR); lseek(fd, (off_t)0, SEEK_END); write(fd, outbuf, OBSIZE);

38. Deleting files #include < unistd.h > int unlink(const char *path); #include < stdio.h > int remove(const char *path); return 0 for success -1 for failure

39. fcntl – making adjustments SYNOPSIS #include <sys/types.h> #include <unistd.h> #include <fcntl.h> intfcntl(intfildes, intcmd, /* arg */ ...); DESCRIPTION The fcntl() function provides for control over open files. The fildes argument is an open file descriptor. The fcntl() function may take a third argument, arg, whose data type, value and use depend upon the value of cmd. The cmdargument specifies the operation to be performed by fcntl().

40. fcntl The available values for cmd are defined in the header <fcntl.h> and include : .. there's a good-sized list of them. Right now, we care about: F_GETFL - get current file status flags F_SETFL - set file status flags

41. fcntl intarg1; if ((arg1=fcntl(fd, F_GETFL)) == -1) printf("filestatus failed \n"); ... switch(arg1 & O_ACCMODE){ case O_WRONLY: .. do something break; case O_RDWR: ... do something else break; case O_RDONLY: ... do something else again break; default: } if (arg1 & O_APPEND) printf(" append flag set "); printf("\n"); return(0);