Understanding Operating Systems: Hierarchical Storage Architecture and File Management

1. Rensselaer Polytechnic Institute CSCI-4210 – Operating Systems David Goldschmidt, Ph.D. Operating Systems{week 15a}

2. Hierarchical storage architecture very fast very small volatile managed as part of the filesystem non-volatile very slow very large

3. File management • A File Management System is a set of OS services that supports files and directories for user applications, OS programs, etc. • Data should be organized andavailable in a convenient andefficient manner • Files are the basicbuilding blocks

4. Files • A file is an abstraction that represents user data, OS data, an executable, a device, etc. • A file is simply a sequence of bytes • Actual storage location (via network?) andformat are transparent to users • Storage scheme on disk is also transparent • Typically involves cylinders, tracks, sectors, etc.

5. File attributes • File attributes include: • Human-readable symbolic name • Type (e.g. executable, directory, text, PDF, etc.) • Logical location (i.e. containing directory or path) • Physical location on disk • Size (in bytes) • Protection or security (i.e. permissions) • Timestamps (created, last modified, last accessed)

6. Pathnames • The pathname (or just path) of a file specifies the sequence of directoriesone must traverse to locatethe file • An absolute path startsat the root node • A relative path startsanywhere

7. Links • A link provides a shortcut to a file and may circumvent the given directory hierarchy • A hard link in Unix isindistinguishable fromthe original file • A symbolic link in Unixis merely a shortcut • A Windows shortcutis just a symbolic link

8. Creating and accessing files • File creation requires space allocation • Opening a file returns a handle or file descriptor • Read and write operationsuse the handle and anoffset (or file pointer) • The close operation simplydeletes the handle anddeallocates any memory

9. Deleting files • Deleting a file deallocates all disk spacemarked as in use by the file • But likely does not erase file contents • Deleted files are recoverable until the disk space is used for (and overwritten by) other file(s) • The delete operation also removes the corresponding entry in the containing directory

10. Access methods • An access method describes the mannerand mechanisms by which a process accesses the data in a file • Two common access methods: • Sequential access (open, read, write, close) • Random access (open, read, write, seek, close)

11. Contiguous disk space allocation (i) • In a contiguous disk space allocation scheme, files are allocated to contiguous blocks of disk space

12. A A A B B C C C D A A A C C C D B B B B A A A C C C B B B B D D A A A C C C B B B B D D Contiguous disk space allocation (ii) • Four files allocated contiguously to disk: • File B outgrows its space and is reallocated: • File D outgrows its space and is reallocated: • Defragmentation combines free disk space:

13. Clustered disk space allocation (i) • In a clustered disk space allocation scheme, files are allocated to clusters of disk space on an as needed basis

14. A A A B B C C C D A A A B B C C C D B B A A A B B C C C D B B D Clustered disk space allocation (ii) • Four files allocated contiguously to disk: • File B outgrows its space, so one or more new clusters are allocated: • File D outgrows its space, so one or more new clusters are allocated:

15. Clustered disk space allocation (iii) • File accesses may cross cluster boundaries,causing slower response times

16. Clustered disk space allocation (iv) • Using a linked approach,each cluster has apointer to the next cluster • Using an indexed approach, a singletable maintains pointers to eachindividual cluster

17. Managing free space • Managing free blocks (or free clusters) may follow the same scheme as for a single file • Linked scheme • Indexed approach • Bitmapped approach: • A bitmap is maintained in memory • Each bit corresponds to a block or cluster • A 0 indicates a free block • A 1 indicates the block is in use

18. File allocation table (i) • A file allocation table (FAT) maps logical filesto their physical addresses on a disk • A pure FAT would have an entryfor each sector (e.g. 512 bytes) • To improve performance, clustersof contiguous sectors are used • Clusters range from 4 to 64 sectors(e.g. 2,048 to 32,768 bytes)

19. File allocation table (ii) • Microsoft FAT-32 filesystem maps to clusters • May require linked clusters to house entire FAT

20. Microsoft NTFS (i) • Microsoft’s New Technology File System (NTFS) introduced as part of Windows NT • Each partition has a Master File Table (MFT):

21. Microsoft NTFS (ii) • Storage allocation in NTFS is based on runs,which are sequential consecutive blocks • This reduces the number of pointers required • NTFS also supports: • File compression • File encryption • Journaling (similar to transaction logging)

22. Linux ext2 and ext3 • Similar to NTFS, Linux uses indexed clusters

23. Unix inodes (i) • In Unix, file information is stored in an index node (I-node), which contains: • Mode and type of file • Number of links to the file • File owner’s userid and groupid • Size of file (in bytes) • Last access and last modified times • Number of blocks allocated to the file • Pointers to the first twelve blocks • Pointers to three additional blocks of pointers

24. cluster cluster … cluster cluster single indirect inode 1024 cluster … cluster double indirect inode 1024 inode 1024 … … triple indirect inode 10242 inode 1024 … … Unix inodes (ii) inode 12 1024 10242 10243