Exploring Ext2 File System Scalability: Analysis & Insights

1. SCALABILITY OF EXT2 Yancan Huang, Guoliang Jin May 13, 2008

2. Motivation • Graph for create

3. Motivation • Graph for open

4. Motivation • Same method, different graphs: • Code for create: asmlinkage long sys_creat(const char __user * pathname, int mode) { return sys_open(pathname, O_CREAT | O_WRONLY | O_TRUNC, mode); } • Code for open: sys_open(pathname, O_RDWR); • Why? • Create: where does the time go?

5. OVERVIEW • Motivation • What is scalability of file system • Experiment environment • Setup • Techniques • Benchmark • Create: where does the time go • The file create process • What does ext2_lookup do • What dose ext2_create do • The file open process • Conclusion & Future work

6. WHAT IS SCALABILITY OF FILE SYSTEM • Large File Systems • Large, Sparse Files • Large, Contiguous Files • Large Directories • Large Numbers of Files

7. EXPERIMENT ENVIRONMENT • Setup: UML • Mount our own ext2 file system called ext2k • With 1GB empty virtual disk • Measuring techniques • gettimeofday • long long c; __asm__ __volatile__ (“rdtsc” : “=A” (c)); • Output techniques • printk • write to log on host • Benchmark • Sequential create and open on a 1GB disk

8. OVERVIEW • Motivation • What is scalability of file system • Experiment environment • Setup • Techniques • Benchmark • Create: where does the time go • The file create process • What does ext2_lookup do • What dose ext2_create do • The file open process • Conclusion & Future work

9. CREATE: WHERE DOES THE TIME GO asmlinkage long sys_creat( const char __user * pathname, int mode) { return sys_open(pathname, O_CREAT | O_WRONLY | O_TRUNC, mode); }

10. THE FILE CREATE PROCESS • The process of sys_open sys_open { // fs/open.c do_sys_open { // fs/open.c getname get_unused_fd_flags do_filp_open { // fs/open.c open_namei // fs/namei.c inameidata_to_filp } fsnotify_open fd_install putname } prevent_tail_call }

11. THE FILE CREATE PROCESS • When open_namei meets file create open_namei { …… path_lookup_create lookup_hash open_namei_create if (! error) return 0; …… } 45% 54%

12. THE FILE CREATE PROCESS • The process of lookup_hash lookup_hash { permission __lookup_hash { cached_lookup // always fail for create struct dentry *new = d_alloc dentry = inode->i_op->lookup if (!dentry) // always true for create dentry = new return dentry } }

13. THE FILE CREATE PROCESS • The process of open_namei_create open_namei_create { vfs_create { may_create security_inode_create dir->i_op->create // dir is an inode fsnotify_create } may_open }

14. We are now in ext2 • Thanks to inode->i_op->lookup • Thanks to inode->i_op->create • Going to ext2_lookup • Going to ext2_create

15. WHAT DOES EXT2_LOOKUP DO • The process of ext2_lookup ext2_lookup { ext2_inode_by_name { ext2_find_entry { …… } } iget d_splice_alias }

16. WHAT DOES EXT2_FIND_ENTRY DO • The process of ext2_find_entry ext2_find_entry { do { ext2_get_page ext2_last_byte while () { ext2_match ext2_next_entry } } while () }

17. NOW WHERE WE ARE • When open_namei meets file create open_namei { …… path_lookup_create lookup_hash open_namei_create if (! error) return 0; …… }

18. WHAT DOES EXT2_CREATE DO • The process of ext2_create ext2_create { ext2_new_inode mark_inode_dirty ext2_add_nondir { ext2_add_link d_instantiate } }

19. WHAT DOES EXT2_ADD_LINK DO • The process of ext2_add_link ext2_add_link { for () { ext2_get_page ext2_last_byte while () { ext2_match ext2_rec_len_from_disk } } __ext2_write_begin ext2_commit_chunk }

20. How often these while loop executed

21. Revisit the create graph

22. THE FILE OPEN PROCESS • The process of sys_open sys_open { // fs/open.c do_sys_open { // fs/open.c getname get_unused_fd_flags do_filp_open { // fs/open.c open_namei // fs/namei.c inameidata_to_filp } fsnotify_open fd_install putname } prevent_tail_call }

23. THE FILE OPEN PROCESS • When open_namei meets file open open_namei { …… if (!(flag & O_CREAT)) { error = path_lookup_open(dfd, pathname, lookup_flags(flag), nd, flag); if (error) return error; goto ok; } …… }

24. THE FILE OPEN PROCESS • The process of path_lookup_open path_lookup_open { __path_lookup_intent_open { get_empty_filp do_path_lookup { path_walk { link_path_walk { …… } } } } }

25. THE FILE OPEN PROCESS • The process of link_path_walk link_path_walk { __link_path_open // in the dcache if (fail) { // force real lookup requests dget mntgrt __link_path_open } }

26. Revisit the open graph

27. OVERVIEW • Motivation • What is scalability of file system • Experiment environment • Setup • Techniques • Benchmark • Create: where does the time go • The file create process • What does ext2_lookup do • What dose ext2_create do • The file open process • Conclusion & Future work

28. Conclusion • In create, ext2_lookup makes sure there won’t be two files with the same name, and ext2_add_link performs a similar routine again • The dentry structure of ext2 is linear • Using B-tree to manage this in memory structure would show better performance • Another scalability issue in ext2 is that its inode number is determined when the disk is formatted

29. Future work • Try to use B-tree to manage the dentry structure and test the performance • But the B-tree itself is complex • Try more workload

30. Questions?