1 / 21

Reducing Fsck time using divide and conquer approach

Reducing Fsck time using divide and conquer approach. Presented by: Kamaya Tapadiya Nidhi Ruhatiya Savita Kulkarni Teja Tamboli. Recent News… Main server for Linux kernel source kernel.org suffered file system corruption. It took over a week to repair the file system.

cathal
Télécharger la présentation

Reducing Fsck time using divide and conquer approach

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Reducing Fsck time using divide and conquer approach Presented by: • Kamaya Tapadiya • Nidhi Ruhatiya • Savita Kulkarni • Teja Tamboli

  2. Recent News… • Main server for Linux kernel source kernel.org suffered file system corruption. • It took over a week to repair the file system.

  3. Filesystems are vulnerable to hardware changes. • Disk capacity is growing exponentially. • Disk bandwidth is not increasing at the same pace. • Seek time is relatively much longer. • Per bit error rate is not improving as fast as disk capacity • is growing. • Increased I/O errors means frequent and longer • FileSystem Consistency checK (FSCK).

  4. Fsck - Filesystem Consistency Check • Scandisk : windows :: Fsck : Linux • Repairs filesystem inconsistency caused by hardware • errors, software bugs or other data corruption. • It needs to traverse the entire filesystem looking for • errors and fix them.

  5. Existing Filesystem • Single meta data block corruption needs the entire filesystem to be unmounted and checked. • During this process the data is completely unavailable.

  6. Goals to achieve… • To reduce the fsck time. • To maintain the on disk data structures with no changes to the existing filesystem.

  7. Proposed solution… • To divide the on disk filesystem format into small fault isolated domains – chunks. • Each chunk has a standalone set of metadata (block number space, allocation bitmaps, superblock). • Each chunk is individually repairable. • Namespace and disk space are still shared.

  8. /root/dir/file.txt USER LEVEL KERNEL LEVEL /root/dir/file.txt Fsckable chunks Chunk 1 (metadata) Chunk 2 (metadata) Chunk 3 (metadata) Chunk 4 (metadata) root dir file.txt - - - - - - - - - - - - - - - - - - - - -Whole Device- - - - - - - - - - - - - - - - - - - - -

  9. Implementation Stackable Filesystem Filesystem stacked above another base filesystem.

  10. User Process System calls Mount() Ioctl() Error Codes System Call Interface USER LEVEL V F S KERNEL LEVEL Filesystem data Vnode Interface Error codes Stackable File System Filesystem data and operations Error codes Vnode Interface Lower File System

  11. Implementation Issues 1.Handling Large Files Forward inode number Chunk A (/movie/cd1) Chunk B (/movie.cont/cd1.cont) Backward inode number

  12. 2. Handling Hardlinks • Hard link- a way of referring an inode with different file names. • Cross Chunk reference- source file is in one chunk and the target file needs to be created in other chunk. • In that case continuation inode for the parent directory of the target file is placed in the same chunk as the source file. • Making sure that the target file entry and the source file are in the same chunk.

  13. 3. Inode Number Management • First 4 bits indicates chunk number eg. for chunk 2 (0010) Chunkfs 0010 1001 0011 1100 0110 1111 1011 1100 Basefs 0000 1001 0011 1100 0110 1111 1011 1100

  14. Partial Fsck Chunk File System Chunk 1 Chunk 2 Chunk 2 Chunk 3 Chunk 4 Fsck corruption

  15. To Summarize… • Makes no changes to the on-disk filesystem layout. • Could be ported to support other filesystems with minimum changes to chunkfs. • Reduced downtime due to degraded mode of operation - • Parts of the filesystem can undergo isolated fsck, while others are online and serving requests. • Reduced fsck memory requirements.

  16. References • Kernel Summit 2006 • The 2006 Linux Filesystem Workshop • http://lwn.net/Articles/190224/ • lkml.org/lkml/2006/6/9/476 • The Design Of Unix operating system -Maurice J. Bach • Understanding Linux Kernel -Daniel P. Bovet and Macro Cesati

  17. Achievements… • Won second prize at Techkriti IIT- Kanpur for the event SOFTWARE CORNER. • Won second prize at Cummins College of Engineering for the event INNOVATION’07 in the category of system programming.

  18. Thank you !!!!

  19. Prediction for 3.5 inch hard disks... <--

  20. Reserved fields in inode: • Forward inode number is stored in Osd1.linux1.l_i_reserved1 • Backward inode number is stored in Osd2.linux2.l_i_reserved2 New flags used: • HAS_CONT_INODE (20 ’th BIT) • IS_CONT_INODE (21 ’th BIT)

  21. Software engineering perspective • Implements the spiral model. • Modularity. • Regression testing. • Use of SVN for maintaining code.

More Related