1.7k likes | 2.12k Vues
ecs150 Fall 2007 : Operating System #5: File Systems (chapters: 6.4~6.7, 8). Dr. S. Felix Wu Computer Science Department University of California, Davis http://www.cs.ucdavis.edu/~wu/ sfelixwu@gmail.com. File System Abstraction. Files Directories. System-call interface.
E N D
ecs150 Fall 2007:Operating System#5: File Systems(chapters: 6.4~6.7, 8) Dr. S. Felix Wu Computer Science Department University of California, Davis http://www.cs.ucdavis.edu/~wu/ sfelixwu@gmail.com ecs150, Fall 2007
File System Abstraction • Files • Directories ecs150, Fall 2007
System-call interface Active file entries VNODE Layer or VFS Local naming (UFS) FFS Buffer cache Block or character device driver Hardware ecs150, Fall 2007
dirp = opendir(const char *filename); struct dirent *direntp = readdir(dirp); struct dirent { ino_t d_ino; char d_name[NAME_MAX+1]; }; directory dirent inode file_name dirent inode file_name dirent inode file_name file file file ecs150, Fall 2007
Local versus Remote • System Call Interface • V-node • Local versus remote • NFS or i-node • Stackable File System • Hard-disk blocks ecs150, Fall 2007
File-System Structure • File structure • Logical storage unit • Collection of related information • File system resides on secondary storage (disks). • File system organized into layers. • File control block – storage structure consisting of information about a file. ecs150, Fall 2007
File Disk • separate the disk into blocks • separate the file into blocks as well • paging from file to disk blocks: 4 - 7- 2- 10- 12 How to represent the file?? How to link these 5 pages together?? ecs150, Fall 2007
Bit torrent pieces • 1 big file (X Gigabytes) with a number of pieces (5%) already in (and sharing with others). • How much disk space do we need at this moment? ecs150, Fall 2007
Hard Disk • Track, Sector, Head • Track + Heads Cylinder • Performance • seek time • rotation time • transfer time • LBA • Linear Block Addressing ecs150, Fall 2007
File Disk blocks 0 file block 0 file block 1 file block 2 file block 3 file block 4 4 7 2 10 12 • What are the disadvantages? • disk access can be slow for “random access”. • How big is each block? 64 bytes? 68 bytes? ecs150, Fall 2007
Kernel Hacking Session • This Friday from 7:30 p.m. until midnight.. • 3083 Kemper • Bring your laptop • And bring your mug… ecs150, Fall 2007
A File System partition partition partition b s i-list directory and data blocks d i-node i-node ……. i-node ecs150, Fall 2007
One Logical File Physical Disk Blocks efficient representation & access ecs150, Fall 2007
An i-node A file ??? entries in one disk block Typical: each block 8K or 16K bytes ecs150, Fall 2007
inode (index node) structure • meta-data of the file. • di_mode 02 • di_nlinks 02 • di_uid 02 • di_gid 02 • di_size 04 • di_addr 39 • di_gen 01 • di_atime 04 • di_mtime 04 • di_ctime 04 ecs150, Fall 2007
System-call interface Active file entries VNODE Layer or VFS Local naming (UFS) FFS Buffer cache Block or character device driver Hardware ecs150, Fall 2007
A File System partition partition partition b s i-list directory and data blocks d i-node i-node ……. i-node ecs150, Fall 2007
125 struct ufs2_dinode { 126 u_int16_t di_mode; /* 0: IFMT, permissions; see below. */ 127 int16_t di_nlink; /* 2: File link count. */ 128 u_int32_t di_uid; /* 4: File owner. */ 129 u_int32_t di_gid; /* 8: File group. */ 130 u_int32_t di_blksize; /* 12: Inode blocksize. */ 131 u_int64_t di_size; /* 16: File byte count. */ 132 u_int64_t di_blocks; /* 24: Bytes actually held. */ 133 ufs_time_t di_atime; /* 32: Last access time. */ 134 ufs_time_t di_mtime; /* 40: Last modified time. */ 135 ufs_time_t di_ctime; /* 48: Last inode change time. */ 136 ufs_time_t di_birthtime; /* 56: Inode creation time. */ 137 int32_t di_mtimensec; /* 64: Last modified time. */ 138 int32_t di_atimensec; /* 68: Last access time. */ 139 int32_t di_ctimensec; /* 72: Last inode change time. */ 140 int32_t di_birthnsec; /* 76: Inode creation time. */ 141 int32_t di_gen; /* 80: Generation number. */ 142 u_int32_t di_kernflags; /* 84: Kernel flags. */ 143 u_int32_t di_flags; /* 88: Status flags (chflags). */ 144 int32_t di_extsize; /* 92: External attributes block. */ 145 ufs2_daddr_t di_extb[NXADDR];/* 96: External attributes block. */ 146 ufs2_daddr_t di_db[NDADDR]; /* 112: Direct disk blocks. */ 147 ufs2_daddr_t di_ib[NIADDR]; /* 208: Indirect disk blocks. */ 148 int64_t di_spare[3]; /* 232: Reserved; currently unused */ 149 }; ecs150, Fall 2007
166 struct ufs1_dinode { 167 u_int16_t di_mode; /* 0: IFMT, permissions; see below. */ 168 int16_t di_nlink; /* 2: File link count. */ 169 union { 170 u_int16_t oldids[2]; /* 4: Ffs: old user and group ids. */ 171 } di_u; 172 u_int64_t di_size; /* 8: File byte count. */ 173 int32_t di_atime; /* 16: Last access time. */ 174 int32_t di_atimensec; /* 20: Last access time. */ 175 int32_t di_mtime; /* 24: Last modified time. */ 176 int32_t di_mtimensec; /* 28: Last modified time. */ 177 int32_t di_ctime; /* 32: Last inode change time. */ 178 int32_t di_ctimensec; /* 36: Last inode change time. */ 179 ufs1_daddr_t di_db[NDADDR]; /* 40: Direct disk blocks. */ 180 ufs1_daddr_t di_ib[NIADDR]; /* 88: Indirect disk blocks. */ 181 u_int32_t di_flags; /* 100: Status flags (chflags). */ 182 int32_t di_blocks; /* 104: Blocks actually held. */ 183 int32_t di_gen; /* 108: Generation number. */ 184 u_int32_t di_uid; /* 112: File owner. */ 185 u_int32_t di_gid; /* 116: File group. */ 186 int32_t di_spare[2]; /* 120: Reserved; currently unused */ 187 }; ecs150, Fall 2007
Bittorrent pieces File size: 10 GB Pieces downloaded: 512 MB How much disk space do we need? ecs150, Fall 2007
#include <stdio.h> #include <stdlib.h> int main (void) { FILE *f1 = fopen("./sss.txt", "w"); int i; for (i = 0; i < 1000; i++) { fseek(f1, rand(), SEEK_SET); fprintf(f1, "%d%d%d%d", rand(), rand(), rand(), rand()); if (i % 100 == 0) sleep(1); } fflush(f1); } # ./t # ls –l ./sss.txt ecs150, Fall 2007
An i-node A file ??? entries in one disk block Typical: each block 1K ecs150, Fall 2007
i-node • How many disk blocks can a FS have? • How many levels of i-node indirection will be necessary to store a file of 2G bytes? (I.e., 0, 1, 2 or 3) • What is the largest possible file size in i-node? • What is the size of the i-node itself for a file of 10GB with only 512 MB downloaded? ecs150, Fall 2007
Answer • How many disk blocks can a FS have? • 264 or 232: Pointer (to blocks) size is 8/4 bytes. • How many levels of i-node indirection will be necessary to store a file of 2G (231) bytes? (I.e., 0, 1, 2 or 3) • 12*210 + 28 * 210 + 28 *28 *2 10 +28 *28 *28 *2 10 >? 231 • What is the largest possible file size in i-node? • 12*210 + 28 * 210 + 28 *28 *2 10 +28 *28 *28 *2 10 • 264 –1 • 232 * 210 You need to consider three issues and find the minimum! ecs150, Fall 2007
Answer: Lower Bound • How many pointers? • 512MB divided by the block size (1K) • 512K pointers times 8 (4) bytes = 4 (2) MB ecs150, Fall 2007
Bittorrent pieces File size: 10 GB Pieces downloaded: 512 MB How much disk space do we need? ecs150, Fall 2007
Answer: Upper Bound • In the worst case, EVERY indirection block has at least one entry! • How many indirection blocks? • Single: 1 block • Double: 1 + 28 • Tripple: 1 + 28 + 216 • Total ~ 216 blocks times 1K = 64 MB • 214 times 1K = 16MB (ufs2 inode) ecs150, Fall 2007
Answer (4) • 2 MB ~ 64 MB ufs1 • 4 MB ~ 16 MB ufs2 • Answer: sss.txt ~17 MB • ~16 MB (inode indirection blocks) • 1000 writes times 1K ~ 1MB ecs150, Fall 2007
An i-node A file ??? entries in one disk block Typical: each block 1K ecs150, Fall 2007
A File System partition partition partition b s i-list directory and data blocks d i-node i-node ……. i-node ecs150, Fall 2007
FFS and UFS • /usr/src/sys/ufs/ffs/* • Higher-level: directory structure • Soft updates & Snapshot • /usr/src/sys/ufs/ufs/* • Lower-level: buffer, i-node ecs150, Fall 2007
# of i-nodes • UFS1: pre-allocation • 3% of HD, about < 25% used. • UFS2: dynamic allocation • Still limited # of i-nods ecs150, Fall 2007
di_size vs. di_blocks • ??? ecs150, Fall 2007
One Logical File Physical Disk Blocks efficient representation & access ecs150, Fall 2007
di_size vs. di_blocks • Logical • Physical • fstat • du ecs150, Fall 2007
Extended Attributes in UFS2 • Attributes associated with the File • di_extb[2]; • two blocks, but indirection if needed. • Format • Length 4 • Name Space 1 • Content Pad Length 1 • Name Length 1 • Name mod 8 • Content variable • Applications: ACL, Data Labelling ecs150, Fall 2007
Some thoughts…. • What can you do with “extended attributes”? • How to design/implement? • Should/can we do it “Stackable File Systems”? • Otherwise, the program to manipulate the EA’s will have to be very UFS2-dependent or FiST with an UFS2 optimization option. • Are there any counter examples? • security and performance considerations. ecs150, Fall 2007
A File System partition partition partition b s i-list directory and data blocks d i-node i-node ……. i-node ecs150, Fall 2007
struct dirent { ino_t d_ino; char d_name[NAME_MAX+1]; }; struct stat {… short nlinks; …}; directory dirent inode file_name dirent inode file_name dirent inode file_name file file file ecs150, Fall 2007
root wheel . 2 directory / 2 drwxr-xr-x .. 2 Apr 1 2004 usr 4 3 vmunix 5 root wheel . 4 drwxr-xr-x directory /usr .. 2 4 Apr 1 2004 bin 7 root wheel foo 6 rwxr-xr-x 5 file /vmunix Apr 15 2004 text data kirk staff 6 rw-rw-r-- file /usr/foo Hello World! Jan 19 2004 root wheel . 7 7 drwxr-xr-x .. 4 directory /usr/bin Apr 1 2004 ex 9 8 groff 10 bin bin vi 9 file /usr/bin/vi 9 rwxr-xr-x text data Apr 15 2004 ecs150, Fall 2007
What is the difference? • ln –s /usr/src/sys/sys/proc.h ppp.h • ln /usr/src/sys/sys/proc.h ppp.h ecs150, Fall 2007