How to create a Filesystem on Linux

1. How to create a Filesystem on Linux Hanning Gao Polytechnic University Scholarship for Service Scholarship for Service

2. Overview • What is a filesystem (UNIX/Linux) • VFS, the Linux fileystem interface • ext2, the Linux filesystem • Programming of a Linux filesystem • Demonstration • Q&A Scholarship for Service

3. What is a filesystem • A filesystem is a data structure! • This data structure and it’s interface allows an operating system to record data information. • A filesystem’s basic structures are directory and file. • Every directory or file must have a unique name(full path name), and also a unique identifier, inode • To enable large file storage, filesystem implements link-listed storage • Advanced filesystem have ACL, security options, distributed, networked, etc. Scholarship for Service

4. What is a filesystem (continued) • Must contain a / root directory • Singular, entry point to the filesystem • Resided in block two • Everything is a file • Even directory is a file (you can cat a dir) • There are special files (links, tty, I/O devices, etc) • Access methods • Raw access (character access) • Block access (accessed through system buffer) Scholarship for Service

5. What is a filesystem (continued) • On UNIX, disks are divided into block size chunk. • The first block of a fileystem, is called the super_block. • This super_block contains important filesystem info • Total number of data blocks • Number of inodes (number of files) • Number of free inodes • Number of free data block • Data block and inodes are two different thing! • Each inode represent a file, but a file can occupy multiple data blocks Scholarship for Service

6. What is a filesystem (continued) Superblock Inode blocks … Data blocks … File 1 inode 5 Dir 2 Datablock 9 Inode 5 Datablock 10 Scholarship for Service

7. What is a filesystem (continued) • Original UNIX had filesystem integrated right onto kernel • Later this is found to be not practical with introduction of new filesystems • Older UNIX, through File System Switch (FSS) • This allows multiple filesystems to exist in parallel, but all operates on one single interface • SVR4, SUN’s VFS/vnode interface • Same idea as FSS, but extends FSS further to include: • Clear separation of filesystem independent and dependent layer • networked filesystem support. • Linux, Virtual File System (VFS) • Not the same as SUN’s VFS/vnode interface! Scholarship for Service

8. VFS, the Linux fileystem interface Scholarship for Service

9. VFS, the Linux fileystem interface (continued) • How to integrate new filesystem to linux? • Through loadable Linux modules • module insertion (insmod) • The fs.h • This is the header file that contains the VFS handle • Provides generic uniform filesystem access support • Implement filesystem specific codes to extend the fs.h’s generic access functions where it is needed. • Example, an encrypted filesystem need to extend the file_operation’s file_read() function to enable automatic decryption. Scholarship for Service

10. VFS, the Linux fileystem interface (continued) • Some sets of operations • file_operations • Operate on file, directory • llseek, read, write, readdir, poll, ioctl, lock, etc • inode_operations • Operate on inodes • create, mkdir, rmdir, lookup, link, unlink, rename, etc • super_operations • Operate on the super block • read_inode, write_inode, delete_inode, statfs, etc • address_space_operations • File I/O with buffer & page cache (page mapping, block access) • readpage, writepage, prepare_write, etc Scholarship for Service

11. VFS, the Linux filesystem interface (continued) • File access can become cumbersome if access is frequent and random • Lots of overhead for finding the file, especially if file is used over and over • Buffer cache and page cache • Buffer cache interact with the disk • Page cache interact with the VFS layer, interact with buffer cache • Used for reading and writing blocks of data • Used to hold dentry (directory cache) • When files/directories are accessed, it goes through page cache first. Scholarship for Service

12. ext2, the Linux filesystem • Support of a single file size up to 64TB • Ext2 internals divided into group • Super block, Group descriptor, Block descriptor, inode bitmap, inode table, data blocks • Different block size length • 1KB, 2KB, 4KB, 8KB • Root has 5% of space reserved • Enable filesystem recovery Scholarship for Service

13. ext2, the Linux filesystem (continued) Struct ext2_super_block{ unsigned long s_inodes_count; unsigned long s_blocks_count; unsigned long s_free_blocks_count; … unsigned long s_mtime; // mount time unsigned long s_wtime; // write time unsigned short s_magic; // the magic # … }; Scholarship for Service

14. ext2, the Linux filesystem (continued) • The inode contains important information regarding to the file • The user/group info, access mode (rwx), size, access time, where the file is located on the disk • Each inode has 15 pointers to data block. • First thirteen points to data block. • Fourteen is called indirect pointer • Fifteen is called doubly indirect pointer Scholarship for Service

15. ext2, the Linux filesystem (continued) Scholarship for Service

16. ext2, the Linux filesystem (continued) • Directory is a file that lists all the files that reside in the directory, with the following information: • The file name • The inode number • The length of the file name • On older UNIX you can actually cat a directory to view the contents Scholarship for Service

17. Programming of a Linux filesystem • To implement a Linux filesystem: • Create a Linux filesystem loadable module • Use the VFS’s to implement the filesystem’s specific access code Scholarship for Service

18. Programming of a Linux filesystem (continued) • To make it a loadable Linux kernel module, there’s three things that need to be done: • A declaration giving information about the type of the module • A function to be called when the module is loaded (load and register filesystem to kernel) • A function to be called when the module is unloaded (cleanup and unregister the filesystem) Scholarship for Service

19. Programming of a Linux filesystem (continued) Static DECLARE_FSTYPE_DEV (myfs_fs_type, “myfs”, myfs_read_super); Static int __init init_myfs_fs(void) { return register_filesystem(&myfs_fs_type); } Static void __exit exit_myfs_fs(void) { unregister_filesystem(&myfs_fs_type); } module_init(init_myfs_fs) module_exit(exit_myfs_fs) Scholarship for Service

20. Programming of a Linux filesystem (continued) • In order to use the filesystem, we need to mount it. The function that supports the mount and unmount are the: • myfs_read_super(struct super_block *, void *data, int ) • This function initialize the filesystem’s magic number, set the block size, the operations associated with filesystem, and the reference to the root inode of this filesystem • myfs_put_super(struct super_block *) • This function finish writing everything, and free the buffer cache and page cache since we do not use it anymore. Scholarship for Service

21. Programming of a Linux filesystem (continued) • Directory access • To enable directory access, the combination of file_operations and inode_operations need to be implemented. • Functions such as read(), readdir(), lookup(), create(), mkdir(), rmdir() will need to be implemented. Scholarship for Service

22. Programming of a Linux filesystem (continued) • File access • Can use generic_file_read(), generic_file_write() operations • Can also implement address_space_operations() for block access (Optimization) Scholarship for Service

23. Demonstration Scholarship for Service

24. Resources • Unix Filesystems by Steve Pate • Filesystem source code • Located in /usr/src/linux-version/fs/ • www.kernelhacking.org • http://inglorion.net/documents/tutorials/tutorfs • http://maria.utcluj.ro/~rbb/files/my_work/os/sfsimpl.html Scholarship for Service

25. Q&A • Questions, comments? Scholarship for Service