CPS110: I/O and file systems

1. CPS110: I/O and file systems Landon Cox April 13, 2009

2. Virtual/physical interfaces Applications Single interface Many types of devices Bytes Blocks File names Disk sectors Directory tree Flat name space “Fast” Slow Consistency Corruptions OS Hardware

3. Device independence • Problem: many different disks • Also interfaces, controllers, etc • Each has their own custom interface • How do we manage this diversity? • Add an abstraction inside the OS

4. Device drivers Rest of OS (can deal with device categories like network card, video card, keyboard, mouse, etc.) OS file systems don’t have to know about specific devices Device drivers Make life easier for the OS (still run in the kernel) Millions of specific devices

5. Device drivers Rest of OS (can deal with device categories like network card, video card, keyboard, mouse, etc.) 70% of Linux kernel code 120,000 driver versions for XP 85% of XP failures Device drivers Millions of specific devices Drivers in Windows vs Mac? In Vista, many drivers run in user space. Why?

6. Crashes in Vista

7. Other abstractions we’ve seen • Abstractions simplify things for their users • Threads • Programmers don’t have to worry about sharing the CPU • Address spaces • Applications don’t have to worry about sharing phys mem • TCP • Network code needn’t worry about unreliable network • Device drivers • Parts of OS don’t have to worry about device variations

8. Virtual/physical interfaces Applications Single interface Many types of devices Bytes Blocks File names Disk sectors Directory tree Flat name space “Fast” Slow Consistency Corruptions OS Hardware

9. Block-oriented vs byte-oriented • Disks are accessed in terms of blocks • Also called sectors • Similar idea to memory pages • E.g. 4KB chunks of data • Programs deal with bytes • E.g. change ‘J’ in “Jello world” to ‘H’

10. Block-oriented vs byte-oriented • How to read less than a block? • Read entire block • Return the right portion • How to write less than a block? • Read entire block • Modify the right portion • Write out entire block • Nothing analogous to byte-grained load/store • Flash devices are even more complicated

11. Disk geometry

12. Surface organized into tracks

13. Parallel tracks form cylinders

14. Tracks broken up into sectors

15. Disk head position

16. Rotation is counter-clockwise

17. About to read a sector

18. After reading blue sector

19. Red request scheduled next After BLUE read

20. Seek to red’s track After BLUE read Seek for RED SEEK

21. Wait for red sector to reach head After BLUE read Seek for RED Rotational latency SEEK ROTATE

22. Read red sector After BLUE read Seek for RED Rotational latency After RED read SEEK ROTATE

23. To access a disk • Queue (wait for disk to be free) • 0-infinity ms • Position disk head and arm • Seek + rotation • 0-12 ms • Pure overhead • Access disk data • Size/transfer rate (e.g. 49 MB/s) • Useful work

24. Goals for optimizing performance • Disk is really slow • In the time to do a disk seek (~6 ms) • 3GHz processor executes 18 million cycles! • Efficiency = transfer time/(positioning+transfer time) • Best option is to eliminate disk I/O (caching) • Then try to minimize positioning time (seeks) • Then try to amortize positioning overhead

25. Goals for optimizing performance • For each re-positioning, transfer a lot of data • True of any storage device • Particularly true for disks • Fast transfer rate, long positioning time • For Seagate Barracuda 1181677LW • Can get 50% efficiency • Transfer 6*49 = 294 KB after 6 ms seek

26. How to reduce positioning time • Can optimize when writing to disk • Try to reduce future positioning time • Put items together that will be accessed at the same time • How can we know this? • Use general usage patterns • Blocks in same files usually accessed together • Files in a directory tend to be accessed together • Files tend to be accessed with the containing directory • Learn from past accesses to specific data • Can also optimize when reading from disk

27. Disk scheduling • Idea • Take a bunch of accesses • Re-order them to reduce seek time • Kind of like traveling salesman problem • Can do this either in OS or on disk • OS knows more about the requests • Disk knows more about disk geometry

28. First-come, first-served (FCFS) • E.g. start at track 53 • Other queued requests • 98, 183, 37, 122, 14, 124, 65, 67 • Total head movement • 640 tracks • Average 80 tracks per seek

29. Shortest seek time first (SSTF) • E.g. start at track 53 • Other requests • 53, 65, 67, 37, 14, 98, 122, 124, 183 • Total head movement • 236 tracks • Average 30 tracks per seek Any problems with this? Starvation

30. Course administration • Project 3 • Due April 22nd • Should have everything you need • Other upcoming dates • Final lecture on April 20th (in one week) • Last day of classes is April 22nd • Final exam will be Wednesday, Apr. 29th, 7p-10p

31. Virtual/physical interfaces Applications Single interface Many types of devices Bytes Blocks File names Disk sectors Directory tree Flat name space “Fast” Slow Consistency Corruptions OS Hardware

32. File systems • What is a file system? • OS abstraction that makes disks easy to use • Place to put persistent data • File system issues • How to map file space onto disk space • Structure and allocation: like mem management • How to use names instead of sectors • Naming and directories: not like memory • But very similar to DNS

33. Intro to file system structure • Overall question: • How do we organize things on disk? • Really a data structure question • What data structures do we use on disk?

34. Intro to file system structure • Need an initial object to get things going • In file systems, this is a file header • Unix: this is called an inode (indexed node) • File header contains info about the file • Size, owner, access permissions • Last modification date • Many ways to organize headers on disk • Use actual usage patterns to make good decisions

35. File system usage patterns • 80% of file accesses are reads (20% writes) • Ok to save on reads, even if it hurts writes • Most file accesses are sequential and full • Form of spatial locality • Put sequential blocks next to each other • Can pre-fetch blocks next to each other • Most files are small • Most bytes are consumed by large files

36. 1) Contiguous allocation • Store a file in one contiguous segment • Sometimes called an “extent” • Reserve space in advance of writing it • User could declare in advance • If grows larger, move it to a place that fits

37. 1) Contiguous allocation • File header contains • Starting location (block #) of file • File size (# of blocks) • Other info (modification times, permissions) • Exactly like base and bounds memory

38. 1) Contiguous allocation • Pros • Fast sequential access • Easy random access • Cons • External fragmentation • Internal fragmentation • Hard to grow files

39. 2) Indexed files • File header • Looks a lot like a page table

40. 2) Indexed files • Pros • Easy to grow (don’t have to reserve in advance) • Easy random access • Cons • How to grow beyond index size? • Sequential access may be slow. Why? • May have to seek after each block read Why wasn’t sequential access a problem with page tables? Memory doesn’t have seek times.

41. 2) Indexed files • Pros • Easy to grow (don’t have to reserve in advance) • Easy random access • Cons • How to grow beyond index size? • Lots of seeks for sequential access How to reduce seeks for sequential access? Don’t want to pre-allocate blocks.

42. 2) Indexed files • Pros • Easy to grow (don’t have to reserve in advance) • Easy random access • Cons • How to grow beyond index size? • Lots of seeks for sequential access How to reduce seeks for sequential access? When you allocate a new block, choose one near the one that precedes it. E.g. blocks in the same cylinder.

43. What about large files? • Could just assume it will be really large • Problem? • Wastes space in header if file is small • Max file size is 4GB • File block is 4KB  1 million pointers • 4 MB header for 4 byte pointers • Remember most files are small • 10,000 small files  40GB of headers

44. What about large files? • Could use a larger block size • Problem? • Internal fragmentation (most files are small) • Solution • Use a more sophisticated data structure

45. 3) Multi-level indexed files • Think of indexed files as a shallow tree • Instead could have a multi-level tree • Level 1 points to level 2 nodes • Level 2 points to level 3 nodes • Gives us big files without wasteful headers

46. 3) Multi-level indexed files Level 1 Level 2 Level 3 (data) How many access to read one block of data? 3 (one for each level)

47. 3) Multi-level indexed files Level 1 Level 2 Level 3 (data) How to improve performance? Caching.

48. 3) Multi-level indexed files • To reduce number of disk accesses • Cache level 1 and level 2 nodes • Often a useful combination • Indirection for flexibility • Caching to speed up indirection • Can cache lots of small pointers • Where else have we seen this strategy? • DNS

49. 3) Multi-level indexed files Level 1 Level 2 Level 3 What about small files (i.e. most files)?

50. 3) Multi-level indexed files Use a non-uniform tree