Understanding Memory Management in Windows 2000: Key Concepts and Operations

1. Lecture 10 Memory Management(2) xlanchen@04/22/2005

2. Contents • Memory Manager Components • Services the Memory Manager Provides • System Memory Pools • Address Space Layout • Address Translation • Page Fault Handling • Virtual Address Descriptors • Working Sets • Page Frame Number Database • Section Objects Understanding the Inside of Windows2000

3. Page Fault Handling • Page fault | |Kernel trap handler | |Memory manager: fault handler Understanding the Inside of Windows2000

4. Page fault conditions • Page fault can be caused by a variety of conditions • See table 7-14 Understanding the Inside of Windows2000

5. 4 basic invalid PTEs • Page file • The desired page resides within a paging file. An in-page operation is initiated Understanding the Inside of Windows2000

6. 4 basic invalid PTEs • Demand zero • A page of zeros • The pager : zero page list | |free list | |standby list • PTE format : zeros the page file number and offset in the former format Understanding the Inside of Windows2000

7. 4 basic invalid PTEs • Transition • The desired page is in memory • the standby, modified, or modified-no-write list • The page is removed from the list and added to the working set Understanding the Inside of Windows2000

8. 4 basic invalid PTEs • Unknown • The PTE is zero, or • The page table doesn't yet exist. Understanding the Inside of Windows2000

9. Prototype PTEs • For shared pages • Prototype page table entries • Section objects : an array of prototype PTEs • First reference: • Prototype PTE  real PTE • Reference counter in PFN database • 0invalid the shared page Understanding the Inside of Windows2000

10. Prototype PTEs • Structure of an invalid PTE that points to the prototype PTE • States of a shared page • Active/valid; transition; modified-no-write;demand zero; page file; mapped file • Lazy updating Understanding the Inside of Windows2000

11. Two virtual pages (valid/invalid) Understanding the Inside of Windows2000

12. In-Paging I/O • Reading a file to satisfy a page fault • In-page I/O operation • Synchronous • not interruptible by APC Understanding the Inside of Windows2000

13. Collided page faults • The currently being in-paged page is faulted by another thread or process • A wait operation is issued • When the I/O completes, all threads will be satisfied Understanding the Inside of Windows2000

14. Page files • Page files • To store modified pages because of modified page writing • 2K supports up to 16 paging files • EXPERIMENT • Viewing system page files • Viewing Page File Usage with Task Manager Understanding the Inside of Windows2000

15. Virtual Address Descriptors • Lazy evaluation • Copy-on-write • Demand paging • Page table constructing • Advantages and disadvantages Understanding the Inside of Windows2000

16. Virtual Address Descriptors Understanding the Inside of Windows2000

17. EXPERIMENT • Viewing Virtual Address Descriptors Understanding the Inside of Windows2000

18. Working Sets • Working Set • A subset of virtual pages resident in physical memory • Two types • Process working sets • System working set Understanding the Inside of Windows2000

19. Paging Policies • How (or when) paging is performed • fetch policy • placement policy • replacement policy Understanding the Inside of Windows2000

20. Fetch policy • When • Eager-paging (Before needed) • Demand-paging (Until needed) • 2K • Demand-paging + …. • to minimize the number of paging I/Os Understanding the Inside of Windows2000

21. Page Fault Read Clustering Values Understanding the Inside of Windows2000

22. Placement policy • Where • the size of CPU memory caches must be considered • to minimize unnecessary thrashing of the cache Understanding the Inside of Windows2000

23. Replacement policy • Which • Which virtual page must be removed from memory to make room for the new page • Include • LRU • FIFO • … • Global or local Understanding the Inside of Windows2000

24. 2K replacement policy • For multiprocessor system • a variation of a local FIFO • For uniprocessor system • Similar to LRU Understanding the Inside of Windows2000

25. Working Set Management • Default Minimum and Maximum Working Set Sizes • To change: • SetProcessWorkingSetSize Maximum value must < system value MmMaximumWorkingSetSize Understanding the Inside of Windows2000

26. Working set management • When paging, it examines • the process's working set limits • and the amount of free memory • When MmAvailablePages <= MmMinimumFreePages • Must trimming working set (working set manager is called) • Working set manager determines • which • how many Understanding the Inside of Windows2000

27. Working Set-Related System Control Variables Understanding the Inside of Windows2000

28. EXPERIMENT • Viewing Process Working Set Sizes • Viewing the Working Set List Understanding the Inside of Windows2000

29. Balance Set Manager and Swapper • Balance set manager • A system thread for • Working set expansion and trimming • Routine: KeBalanceSetManager • Two event • 1-second timer expiring • Internal working set manager Understanding the Inside of Windows2000

30. Balance set manager • Every fourth time wakes up swapper • Swapper: routine (KeSwapProcessOrStack) • Checks look-aside lists and adjusts their depths • Looks for priority boosted threads • Calls the working set manager Understanding the Inside of Windows2000

31. Swapper • Called by • Balance set manager • Or scheduling code • Looks for threads that can be swapped out • And swapped in some threads Understanding the Inside of Windows2000

32. System Working Set • five kinds of pages • System cache pages • Paged pool • Pageable code and data in Ntoskrnl.exe • Pageable code and data in device drivers • System mapped views (sections mapped at 0xA0000000, such as Win32k.sys) Understanding the Inside of Windows2000

33. System Working Set Performance Counters Understanding the Inside of Windows2000

34. System Working Set • Minimum and Maximum Size Understanding the Inside of Windows2000

35. Page Frame Number Database • the page frame number (PFN) database describes the state of each page in physical memory. • States include • Active/Transition/Standby/Modified/Modified no-write/Free/Zeroed/Bad • The PFN database consists of an array of structures that represent each physical page of memory on the system. Understanding the Inside of Windows2000

36. Page tables & PFN database Understanding the Inside of Windows2000

37. Page lists in PFN database Understanding the Inside of Windows2000

38. EXPERIMENT • Viewing the PFN Database Understanding the Inside of Windows2000

39. Page List Dynamics state diagram for page frame transitions Understanding the Inside of Windows2000

40. EXPERIMENT • Viewing Page Fault Behavior Understanding the Inside of Windows2000

41. Modified Page Writer • writing pages back to disk, include two system threads • MiModifiedPageWriter • To paging file • MiMappedPageWriter • To mapped files • The modified page writer event is triggered by • the number of modified pages > MmModifiedPageMaximum • Or MmAvailablePages < MmMinimumFreePages Understanding the Inside of Windows2000

42. Modified Page Writer Values Understanding the Inside of Windows2000

43. PFN Data Structures • The states of a PFN entry Understanding the Inside of Windows2000

44. Modified state Prototype PTE Parity error Read in progress Write in progress Start of nonpaged pool End of nonpaged pool In-page error Flags Within PFN Database Entries Understanding the Inside of Windows2000

45. Section Objects • section object can be mapped to the paging file or to another file on disk. • Section objects, like other objects, are allocated and deallocated by the object manager Understanding the Inside of Windows2000

46. A section object Understanding the Inside of Windows2000

47. Internal section structures Understanding the Inside of Windows2000

48. EXPERIMENT • Viewing Section Objects • Viewing Control Areas Understanding the Inside of Windows2000