Exploiting Gray-Box Knowledge in Buffer Cache Management for Improved I/O Performance

1. Exploiting Gray-Box Knowledge of Buffer Cache Management Nathan C. Burnett, John Bent, Andrea C. Arpaci-Dusseau, Remzi H. Arpaci-Dusseau University of Wisconsin - Madison Department of Computer Sciences

2. Caching • Buffer cache impacts I/O performance • Cache hits much faster than disk reads OS Without Cache Knowledge: 2 disk reads Buffer Cache With Cache Knowledge 1 disk read Data Blocks

3. Knowledge is Power • Applications can use knowledge of cache state to improve overall performance • Web Server • Database Management Systems • Often no interface for finding cache state • Abstractions hide information

4. Workload + Policy  Contents • Cache contents determined by: • Workload • Replacement policy • Algorithmic Mirroring • Observe workload • Simulate cache using policy knowledge • Infer cache contents from simulation model

5. Gaining Knowledge • Application knows workload • Assume application dominates cache • Cache policy is usually hidden • Documentation can be old, vague or incorrect • Source code may not be available • How can we discover cache policy?

6. Policy Discovery • Fingerprinting: automatic discovery of algorithms or policies (e.g. replacement policy, scheduling algorithm) • Dust - Fingerprints buffer cache policies • Correctly identifies many different policies • Requires no kernel modification • Portable across platforms

7. This Talk • Dust • Detecting initial access order (e.g. FIFO) • Detecting recency of access (e.g. LRU) • Detecting frequency of access (e.g. LFU) • Distinguishing clock from other policies • Fingerprints of Real Systems • NetBSD 1.5, Linux 2.2.19, Linux 2.4.14 • Exploiting Gray-Box Knowledge • Cache-Aware Web Server • Conclusions & Future Work

8. Dust • Fingerprints the buffer cache • Determines cache size • Determines cache policy • Determines cache history usage • Manipulate cache in controlled way • open/read/seek/close

9. Replacement Policies • Cache policies often use • access order • recency • frequency • Need access pattern to identify attributes • Explore in simulation • Well controlled environment • Variety of policies • Known implementations

10. Dust • Move cache to known state • Sets initial access order • Sets access recency • Sets frequency • Cause part of test data to be evicted • Sample data to determine cache state • Read a block and time it Repeat for confidence

11. Setting Initial Access Order Test Region Eviction Region for ( 0 test_region_size/read_size) { read(read_size); }

12. FIFO Priority Newer Pages Older Pages FIFO gives latter part of file priority

13. Detecting FIFO Out of Cache In Cache • FIFO evicts the first half of test region

14. Setting Recency Test Region Eviction Region Right Pointer Left Pointer do_sequential_scan(); left = 0; right = test_region_size/2; for ( 0 test_region_size/read_size){ seek(left); read(read_size); seek(right); read(read_size); right+=read_size; left+= read_size; }

15. LRU Priority LRU gives priority to 2nd and 4th quarters of test region

16. Detecting LRU • LRU evicts 1st and 3rd quarters of test region

17. Setting Frequency Test Region Eviction Region 2 3 4 5 6 6 5 4 3 2 7 Right Pointer Left Pointer do_sequential_scan(); left = 0; right = test_region_size/2; left_count = 1; right_count = 5; for ( 0  test_region_size/read_size) for (0  left_count) seek(left); read(read_size); for (0  right_count) seek(right); read(read_size); right+=read_size; left+= read_size; right_count++; left_count--;

18. LFU Priority LFU gives priority to center of test region

19. Detecting LFU • LFU evicts outermost stripes • Two stripes partially evicted

20. The Clock Algorithm • Used in place of LRU • Ref. bit set on reference • Ref. bit cleared as hand passes • Hand replaces a page with a ref. bit that’s already clear • On eviction, hand searches for a clear ref. bit Page Frame Reference bit

21. Detecting Clock Replacement • Two pieces of initial state • Hand Position • Reference Bits • Hand position is irrelevant – circular queue • Dust must control for reference bits • Reference bits affect order of replacement

22. Detecting Clock Replacement • Uniform reference bits • Random reference bits

23. Clock - Reference Bits Matter • Two fingerprints for Clock • Ability to produce both will imply Clock • Need a way to selectively set reference bits • Dust manipulates reference bits • To set bits, reference page • To clear all bits, cause hand to sweep • Details in paper

24. Dust Summary • Determines cache size (needed to control eviction) • Differentiates policies based on • access order • recency • frequency • Identifies many common policies • FIFO, LRU, LFU, Clock, Segmented FIFO, Random • Identifies history-based policies • LRU-2, 2-Queue

25. This Talk • Dust • Detecting initial access order (e.g. FIFO) • Detecting recency of access (e.g. LRU) • Detecting frequency of access (e.g. LFU) • Distinguishing clock from other policies • Fingerprints of Real Systems • NetBSD 1.5, Linux 2.2.19, Linux 2.4.14 • Exploiting Gray-Box Knowledge • Cache-Aware Web Server • Conclusions & Future Work

26. Fingerprinting Real Systems • Issues: • Data is noisy • Policies usually more complex • Buffer Cache/VM Integration • Cache size might be changing • Platform: • Dual 550 MHz P-III Xeon, 1GB RAM, Ultra2 SCSI 10000RPM Disks

27. F I F O NetBSD 1.5 L R U L F U • Increased variance due to storage hierarchy

28. F I F O NetBSD 1.5 L R U L F U • Four distinct regions of eviction/retention

29. F I F O NetBSD 1.5 L R U L F U • Trying to clear reference bits makes no difference • Conclusion: LRU

30. F I F O Linux 2.2.19 L R U L F U • Very noisy but looks like LRU • Conclusion: LRU or Clock

31. F I F O Linux 2.2.19 L R U L F U • Clearing Reference bits changes fingerprint • Conclusion: Clock

32. F I F O Linux 2.4.14 L R U L F U • Low recency areas are evicted • Low frequency areas also evicted • Conclusion: LRU with page aging

33. This Talk • Dust • Detecting initial access order (e.g. FIFO) • Detecting recency of access (e.g. LRU) • Detecting frequency of access (e.g. LFU) • Distinguishing clock from other policies • Fingerprints of Real Systems • NetBSD 1.5, Linux 2.2.19, Linux 2.4.14 • Exploiting Gray-Box Knowledge • Cache-Aware Web Server • Conclusions & Future Work

34. Algorithmic Mirroring • Model Cache Contents • Observe inputs to cache (reads) • Use knowledge of cache policy to simulate cache • Use model to make application-level decisions

35. NeST • NeST - Network Storage Technology • Software based storage appliance • Supports HTTP, NFS, FTP, GridFTP, Chirp • Allows configurable number of requests to be serviced concurrently • Scheduling Policy: FIFO

36. Cache-Aware NeST • Takes policy & size discovered by Dust • Maintains algorithmic mirror of buffer cache • Updates mirror on each request • No double buffering • May not be a perfect mirror • Scheduling Policy: In-Cache-First • Reduce latency by approximating SJF • Improve throughput by reducing disk reads

37. Performance 144 clients randomly requesting 200, 1MB files Server: P-III Xeon, 128MB Clients: 4 X P-III Xeon, 1GB Gigabit Ethernet Linux 2.2.19 • Improvement in response time • Robust to inaccuracies in cache estimate

38. Summary • Fingerprinting • Discovers OS algorithms and policies • Dust • Portable, user-level cache policy fingerprinting • Identifies FIFO, LRU, LFU, Clock, Random, 2Q, LRU-2 • Fingerprinted Linux 2.2 & 2.4, Solaris 2.7, NetBSD 1.5 & HP-UX 11.20 • Algorithmic Mirroring • Keep track of kernel state in user-space • Use this information to improve performance • Cache-Aware NeST • Uses mirroring to improved HTTP performance

39. Future Work • On-line, adaptive detection of cache policy • Policy manipulation • Make other applications cache aware • Databases • File servers (ftp, NFS, etc.) • Fingerprint other OS components • CPU scheduler • filesystem layout

40. Questions?? • Gray-Box Systems • http://www.cs.wisc.edu/graybox/ • Wisconsin Network Disks • http://www.cs.wisc.edu/wind/ • NeST • http://www.cs.wisc.edu/condor/nest/

41. F I F O Solaris 2.7 L R U L F U

42. F I F O HP-UX 11.20 (IPF) L R U L F U • Low recency areas are evicted • Low frequency areas also evicted • Conclusion: LRU with page aging

43. Related Work • Gray-Box (Arpaci-Dusseau) • Cache content detector • Connection Scheduling (Crovella, et. al.) • TBIT (Padhye & Floyd)

44. Clock - Uniform Reference Bits File Buffer Cache before test scan • After initial scan, cache state does not change • First half of test region is evicted Buffer Cache after test scan, before eviction scan

45. Clock - Random Reference Bits File Buffer Cache before test scan • Initial Sequential Scan • Test scan does not change cache state Buffer Cache after test scan, before eviction scan

46. Manipulating Reference Bits Buffer Cache after touching all resident data • Setting bits is easy • Clear bits by causing hand to do a circuit Buffer Cache after an additional small read