1 / 55

Lecture 15 File Systems

Lecture 15 File Systems. xlanchen@06/03/2005. Contents. Windows 2000 File System Formats NTFS Design Goals and Features File System Driver Architecture NTFS File System Driver NTFS On-Disk Structure. Windows 2000 File System Formats. CDFS (CD-ROM File System)

dominy
Télécharger la présentation

Lecture 15 File Systems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Lecture 15 File Systems xlanchen@06/03/2005

  2. Contents • Windows 2000 File System Formats • NTFS Design Goals and Features • File System Driver Architecture • NTFS File System Driver • NTFS On-Disk Structure Understanding the Inside of Windows2000

  3. Windows 2000 File System Formats • CDFS (CD-ROM File System) • 1988, read-only formatting standard for CD-ROM media • UDF • FAT12, FAT16, and FAT32 • NTFS • the native file system format of Windows 2000 Understanding the Inside of Windows2000

  4. FAT series (12, 16, 32) • FAT format organization • Example: file allocation table Understanding the Inside of Windows2000

  5. NTFS Design Goals • Recoverability • Security • Data Redundancy • Fault Tolerance Understanding the Inside of Windows2000

  6. Multiple data streams Unicode-based names General indexing facility Dynamic bad-cluster remapping Hard links and junctions Compression and sparse files Change logging Per-user volume quotas Link tracking Encryption POSIX support Defragmentation NTFS Features Understanding the Inside of Windows2000

  7. NTFS files • NTFS file = set of ($attribute-name, data) pairs • Std attributes include: • FileName, flags, data, MSDOS-name, ACL, other… • Filename up to 255 characters • Values (data) held in MFT entry if possible • Otherwise: • Data attribute = set of (start-VCN, start-LCN, #clusters) • Allows sequence of VCNs to be discovered • Provides VCN->LCN->cluster mapping • “attribute list” attribute added if MFT entry too small • Points to (first) overflow MFT rec for mappings Understanding the Inside of Windows2000

  8. NTFS Directories • An index of filenames • Index blocks organized as balanced (b+) tree • Tree pointer gives (VCN,LCN) of next block • Index entries contain: • File Reference Number, plus • Size, timestamp, etc (for directory browsing) (saves reading MFT rec to find file attributes) • NT4 indexes on filename only • NT5 indexes on other file attributes also Understanding the Inside of Windows2000

  9. File System Driver (FSD) Architecture • FSDs manage file system formats • Kernel mode • Two different types of FSD (2K) • Local FSDs • Remote FSDs Understanding the Inside of Windows2000

  10. Local FSDs • A local FSD must register with the I/O manager • Local FSDs include: • Ntfs.sys, Fastfat.sys, Udfs.sys, Cdfs.sys, and the Raw FSD (integrated in Ntoskrnl.exe). Understanding the Inside of Windows2000

  11. Local FSD & other concept of the OS • Boot sector • I/O manager • volume parameter block (VPB) • Storage device  file system device • Cache manager Understanding the Inside of Windows2000

  12. Remote FSDs • Remote FSDs consist of two components: • a client and a server • Client-side remote FSD (2K: LANMan Redirector) • allows applications to access remote files and directories • Accepts I/O request & translates into network commands • Server-side FSD (2K: LANMan Server) • Listens and fulfills the command Understanding the Inside of Windows2000

  13. Remote FSD operation Understanding the Inside of Windows2000

  14. EXPERIMENT • Viewing the List of Registered File Systems Understanding the Inside of Windows2000

  15. File system operation • Two ways • Directly, file I/O functions • Indirectly, file mapping • An FSD can be invoked through several paths • Explicit file I/O • From the memory manager's modified page writer • Indirectly from the cache manager's lazy writer • Indirectly from the cache manager's read-ahead thread • From the memory manager's page fault handler Understanding the Inside of Windows2000

  16. Components involved in file system I/O Understanding the Inside of Windows2000

  17. NTFS FSD • Components of the Windows 2000 I/O system Layered drivers Understanding the Inside of Windows2000

  18. NTFS and related components Understanding the Inside of Windows2000

  19. Log File Service (LFS) • NTFS provides file system recoverability by means of a transaction-processing technique called logging • LFS is a series of kernel-mode routines inside the NTFS driver Understanding the Inside of Windows2000

  20. NTFS data structures Understanding the Inside of Windows2000

  21. NTFS On-Disk Structure • Volumes • Clusters • Directories • The storage of actual file data and attribute information • NTFS data compression Understanding the Inside of Windows2000

  22. Volumes • A logical partition on a disk • A disk can have one volume or several • stores all file system data as ordinary files, such as • Bitmaps、directories、system bootstrap Understanding the Inside of Windows2000

  23. Clusters • Cluster size established when formatting • Also called cluster factor • 2n sectors • Cluster vs. sector • NTFS is independent from physical sector sizes • LCN, logical cluster numbers • the numbering of all clusters of the volume • VCN, virtual cluster numbers • Numberthe clusters belonging to a particular file Understanding the Inside of Windows2000

  24. Master File Table (MFT) • All data is contained in files, including metadata • Metadata • the data structures used to locate and retrieve files, • the bootstrap data, • the bitmap that records the allocation state of the entire volume • Easy to locate and maintain • Each can be protected by a security descriptor Understanding the Inside of Windows2000

  25. Master File Table (MFT) • MFT, the heart • Implemented as an array of file records • File records, fixed size, 1KB • Logically, contains one record for each file including the MFT itself. • Metadata files (with name prefixed with “$”) • $Mft Understanding the Inside of Windows2000

  26. Metadata files in MFT Understanding the Inside of Windows2000

  27. Mount a volume with MFT • find the physical disk address of the MFT from the boot sector • Find information inside the file record of MFT • Open more metadata file • Perform the file system recovery operation • Open other metadata file Understanding the Inside of Windows2000

  28. Other metadata files • log file ($LogFile) • root directory ("\") • bitmap file ($Bitmap) • security file ($Secure) • boot file ($Boot) • bad-cluster file ($BadClus) • extensions ($Extend), a metadata directory • object identifier file ($ObjId), the quota file ($Quota), the change journal file ($UsnJrnl), and the reparse point file ($Reparse). Understanding the Inside of Windows2000

  29. File record vs. File • Normally, 1:1 • May n:1 • If a file has a large number of attributes • or becomes highly fragmented • First one called base file record • stores the locations of the others • Others extended file record Understanding the Inside of Windows2000

  30. File Reference Numbers • A file on an NTFS volume is identified a file reference • 64-bit • File number, index to the file's file record position in the MFT • Sequence number, the reused times of an MFT file record position Understanding the Inside of Windows2000

  31. File Records • File, a collection of attribute/value pairs • Filename • time stamp information • unnamed data attribute • additional named data attributes Understanding the Inside of Windows2000

  32. Understanding the Inside of Windows2000

  33. Understanding the Inside of Windows2000

  34. Attribute streams • Each file attribute is stored as a separate stream of bytes within a file • Streams is the unit of file operation • create, delete, read and write • Attribute type code • The file attributes in an MFT record are ordered by type codes • Attribute • Type code: value: optional name Understanding the Inside of Windows2000

  35. Filenames • NTFS, <=255 characters Understanding the Inside of Windows2000

  36. Resident and Nonresident Attributes • resident attribute • the value of an attribute is stored directly in the MFT • Example: Understanding the Inside of Windows2000

  37. Several attributes are defined as always being resident • The standard information • Index root attributes Understanding the Inside of Windows2000

  38. Resident attribute header and value • Example: filename attribute • Only need once disk accessing Understanding the Inside of Windows2000

  39. MFT file record for a small directory • If a particular attribute is too large • Clusters outside MFT is allocated, called run • If the value size grows, more runs is allocated • This is nonresident attributes Understanding the Inside of Windows2000

  40. Resident or nonresident • Determined by the file system • Location transparent to the process • Example: • MFT file record for a large file with two data runs Understanding the Inside of Windows2000

  41. A large directory can also have nonresident attributes • Example: MFT file record for a large directory with a nonresident filename index Understanding the Inside of Windows2000

  42. Keeping track of the runs • VCN-to-LCN mapping pairs • VCN • LCN • Example: • VCNs & LCNs for a nonresident data attribute Understanding the Inside of Windows2000

  43. VCN-to-LCN mappings Understanding the Inside of Windows2000

  44. Indexing • A file directory is simply an index of filenames • Example: • Filename index for a volume's root directory Understanding the Inside of Windows2000

  45. Index root attribute • For large directories: • Index buffer + B+ tree • With index root attribute contains the first level of B+ tree • File index entry: • The file reference in the MFT • Time stamp • File size information Understanding the Inside of Windows2000

  46. The index allocation attribute • For index buffer runs: VCNLCN Understanding the Inside of Windows2000

  47. Data Compression and Sparse Files • NTFS supports compression • per-file, per-directory, or per-volume • Related func • GetVolumeInformation • GetCompressedFileSize • DeviceIoControl Understanding the Inside of Windows2000

  48. Compressing Sparse Data • Sparse data is often large but contains only a small amount of nonzero data relative to its size • Runs of a noncompressed file and the related MFT record Understanding the Inside of Windows2000

  49. Compression technique • to remove long strings of zeros from the file • NTFS allocates space only for runs that contain nonzero data • So certain ranges of the file's VCNs have no disk allocations Understanding the Inside of Windows2000

  50. example • Runs of a compressed file containing sparse data 16~31 64~127 Understanding the Inside of Windows2000

More Related