Linux Kernel Development

1. Linux Kernel Development Chap.12 The Virtual Filesystem Bai,Peng WMN Lab 2007.05.18

2. Intruduce • The Virtual Filesystem (VFS) is the subsystem of the kernel that implements the filesystem-related interfaces provided to user-space programs. • All filesystems rely on the VFS to allow them to coexist and interoperate. • This enables programs to use standard Unix system calls to read and write to different filesystems on different media.

3. Figure1. The VFS in action: Using the cp(1) utility to move data from a hard disk mounted as ext3 to a removable disk mounted as ext2. Two different filesystems, two different media. One VFS.

4. 12.1 Common Filesystem Interface • The VFS is the glue that enables system calls such as open(), read(), write(),copy() and move() to work regardless of the filesystem or underlying physical medium. • In older operating systems (think DOS), this would never have worked. • Modern operating systems abstract access to the filesystems via a virtual interface that such interoperation and generic access is possible. • New filesystems and new varieties of storage media can find their way into Linux, and programs need not be rewritten or even recompiled.

5. 12.2 Filesystem Abstraction Layer(1/2) • Such a generic interface for any type of filesystem is feasible only because the kernel itself implements an abstraction layer around its low-level filesystem interface. • This abstraction layer enables Linux to support different filesystems. This is possible because the VFS provides a common file model that is capable of representing any conceivable filesystem's general features and behavior.

6. 12.2 Filesystem Abstraction Layer(2/2) • The abstraction layer works by defining the basic conceptual interfaces and data structures that all filesystems support. The filesystems mold their view of concepts to match the expectations of the VFS. • The actual filesystem code hides the implementation details. To the VFS layer and the rest of the kernel, however, each filesystem looks the same. • So the kernel easily works with any filesystem and the exported user-space interface seamlessly works on any filesystem.

7. For example, consider a simple user-space program that does write(f, &buf, len); Figure 2. The flow of data from user-space issuing a write() call, through the VFS's generic system call, into the filesystem's specific write method, and finally arriving at the physical media.

8. 12.3 Unix Filesystems(1/3) • Unix has provided four basic filesystem-related abstractions: files, directory entries, inodes, and mount points. • A filesystem is a hierarchical storage of data adhering to a specific structure and contain files, directories, and associated control information. • In Unix, filesystems are mounted at a specific mount point named namespace. This enables all mounted filesystems to appear as entries in a single tree.

9. 12.3 Unix Filesystems(2/3) • A file is an ordered string of bytes with a human-readable name for identification by both the system and the user. • Files are organized in directories. A directory is analogous to a folder and usually contains related files. • "/home/wolfman/butter” the root directory /, the directories home and wolfman, and the file butter are all directory entries, called dentries. • In Unix, directories are actually normal files that simply list the files contained therein.

10. 12.3 Unix Filesystems(3/3) • Inode is used to store associated information of file. • All this information is tied together with the filesystem's control information, which is stored in the superblock. The superblock is a data structure containing information about the filesystem as a whole. • Unix filesystems implement these notions as part of their physical on-disk layout. The Linux VFS is also designed to work with filesystems that understand and implement such concepts.

11. 12.4 VFS Objects and Their Data Structures(1/3) • The VFS is object-oriented. A set of data structures represents the common file model. These data structures are like objects. • The four primary object types of the VFS are: • The superblock object, which represents a specific mounted filesystem. • The inode object, which represents a specific file. • The dentry object, which represents a directory entry, a single component of a path. • The file object, which represents an open file as associated with a process.

12. 12.4 VFS Objects and Their Data Structures(2/3) • An operations object is contained within each of these primary objects. These objects describe the methods that the kernel invokes against the primary objects. • The super_operations object, which kernel can invoke on a specific filesystem, such as read_inode() and sync_fs(). • The inode_operations object, which kernel can invoke on a specific file, such as create() and link(). • The dentry_operations object, which kernel can invoke on a specific directory entry, such as d_compare() and d_delete(). • The file object, which process can invoke on an open file, such as read() and write(). • [note] Objects refer to structures not explicit object data types.

13. 12.4 VFS Objects and Their Data Structures(3/3) • Other VFS Objects • Each registered filesystem is represented by a file_system_type structure. This object describes the filesystem and its capabilities. • Each mount point is represented by the vfsmount structure. This structure contains information about the mount point, such as its location and mount flags. • Three per-process structures describe the filesystem and files associated with a process. They are the file_struct, fs_struct, and namespace structures.

14. The Superblock Object(1/2) • The superblock object is used to store information describing that specific filesystem. • This object usually corresponds to the filesystem superblock or the filesystem control block, which is stored in a special sector on disk . Filesystems that are not disk-based generate the superblock on the fly and store it in memory. • The superblock object is represented by struct super_block and defined in <linux/fs.h>.

16. The Superblock Object(2/2) • Superblock Operations • The most important item in the superblock object is s_op, which is the superblock operations table. • The superblock operations table is represented by struct super_operations and is defined in <linux/fs.h>. • Each item in this structure is a pointer to a function that operates on a superblock object. The superblock operations perform low-level operations on the filesystem and its inodes. • When a filesystem needs to perform an operation on its superblock, it follows the pointers from its superblock object to the desired method.

18. The Inode Object(1/2) • The inode object represents all the information needed by the kernel to manipulate a file or directory. • For Unix-style filesystems, this information is simply read from the on-disk inode. If a filesystem does not have inodes, the filesystem must obtain the information from wherever it is stored on the disk. • The inode object is represented by struct inode and is defined in <linux/fs.h>. • An inode represents each file on a filesystem. This includes special files, such as device files or pipes.

20. The Inode Object(2/2) • Inode Operations • The inode_operations member describes the filesystem's implemented functions that the VFS can invoke on an inode. • inode operations are invoked i->i_op->truncate(i)to a particular inode. • The inode_operations structure is defined in <linux/fs.h>.

22. The Dentry Object (1/4) • As we know VFS treats directories as files. In the path /bin/vi, both bin and vi are all files. • But VFS often needs to perform directory-specific operations which need to translate each component of a path. • A dentry is a specific component in a path. Using the previous example, /, bin, and vi are all dentry objects. The first two are directories and the last is a regular file. Dentries might also include mount points. • Dentry object does not correspond to any sort of on-disk data structure. The VFS creates it on the fly from a string representation of a path name. There is still no flag in struct dentry specifies whether the object is modified.

24. The Dentry Object (2/4) • Dentry State • A used dentry corresponds to a valid inode and indicates that there are one or more users of the object. A used dentry is in use by the VFS and points to valid data and, thus, cannot be discarded. • An unused dentry corresponds to a valid inode, but indicates that the VFS is not currently using the dentry object. Because the dentry object still points to a valid object, the dentry is kept around-cached in case it is needed again. If it is necessary to reclaim memory, the dentry can be discarded because it is not in use. • A negative dentry is not associated with a valid inode because either the inode was deleted or the path name was never correct to begin with. The dentry is kept around for future quick lookups. It can be destroyed if needed.

25. The Dentry Object (3/4) • The Dentry Cache • The kernel caches dentry objects in the dentry cache (dcache) to save resource. • The dentry cache consists of three parts: • Lists of "used" dentries that are linked off their associated inode via the i_dentry field of the inode object. • A doubly linked "least recently used" list of unused and negative dentry objects. The list is insertion sorted by time, such that entries toward the head of the list are newest. kernel must remove entries to reclaim memory from the tail. • A hash table and hashing function used to quickly resolve a given path into the associated dentry object.

26. The Dentry Object (4/4) • Dentry Operations • The dentry_operations structure specifies the methods that the VFS invokes on directory entries on a given filesystem.

27. The File Object • The file object is used to represent a file opened by a process. • Processes deal directly with files, not superblocks, inodes, or dentries. • The file object(but not the physical file) is created by the open() system call and destroyed in response to the close() system call. All these file-related calls are actually methods defined in the file operations table. • Because of multiple processes operating, there can be multiple file objects in existence for the same file. • The file object merely represents a process's view of an open file. The object points back to the dentry (which in turn points back to the inode) that actually represents the open file. The inode and dentry objects are unique.

28. [note] Because the file object does not actually correspond to any on-disk data, there is no flag in the object to represent whether the object is dirty and needs to be written back to disk.

30. 12.5 Data Structures Associated with Filesystems • Except the fundamental VFS objects, the kernel uses other standard data structures to manage data related to filesystems. • The first object is used to describe a specific variant of a filesystem, such as ext3 or XFS. • The second data structure is used to describe a mounted instance of a filesystem.

31. 12.6 Data Structures Associated with a Process • Each process on the system has its own list of open files, root filesystem, current working directory, mount points, and so on. • Three data structures tie together the VFS layer and the processes on the system: • The files_struct is pointed to by the files entry in the processor descriptor. All per-process information about open files and file descriptors is contained therein. • fs_struct contains filesystem information related to a process and is pointed at by the fs field in the process descriptor. This structure holds the current working directory and root directory of the current process. • namespace structure is pointed at by the namespace field in the process descriptor. They enable each process to have a unique view of the mounted filesystems on the system not just a unique root directory, but an entirely unique filesystem hierarchy, if desired.

32. 12.7 Filesystems in Linux • Linux supports a wide range of filesystems, from native filesystems to networked filesystems than 50 filesystems alone in the official kernel. • The VFS layer provides these disparate filesystems with both a framework for their implementation and an interface for working with the standard system calls. • The VFS layer both makes it clean to implement new filesystems in Linux and allows those filesystems to automatically interoperate via the standard Unix system calls.