Memory Leaks

1. Memory Leaks Demand Technology 1020 Eighth Avenue South, Suite 6, Naples, FL 34102 phone: (941) 261-8945 fax: (941) 261-5456 e-mail: markf@demandtech.com http://www.demandtech.com

2. Outline • Topics • Virtual Memory concepts • Virtual Memory constraints in 32-bit Windows • Leaking processes • Page faults/sec

3. Virtual Memory Manager • 4 GB virtual per process address space • Lower 2 GB - Private • Page 0 reserved • Code pages • Heap • Upper 2 GB - System • System code • Shared dlls • System cache ffff ffff16 System 7fff ffff16 User 0000 0000

4. Virtual Memory Manager • All address spaces share the virtual addresses from the same upper 2 GB of the System area • All virtual addresses in the lower 2 GB User area are unique • Shared memory in the System range can be used for IPC ffff ffff16 System Shared 7fff ffff16 User User User 0000 0000

5. Virtual Memory Manager • Boot.ini /3 GB switch • Virtual storage constraint relief for some applications • SQL Server • Exchange 2000 • Etc.

6. AWE • Windows 2000 - Address Windowing Extensions • Address RAM > 4 GB limit -Intel Pentium III, IV • requires PSE36 Driver • Real addresses only • Not backed on paging files • PTEs remain 32-bit

7. Win64 Virtual Memory

8. Virtual Memory Manager • Real Memory allocation Counters • Available Bytes • Pool non-paged Bytes, • Pool Paged Resident Bytes, • System Cache Resident Bytes, • System Code Resident Bytes, • System Driver Resident Bytes • Instantaneous Counters (all reported as Bytes) • Cache Bytes is actually the pageable System Working Set = Pool Paged Resident Bytes + System Cache Resident Bytes + System Code Resident Bytes + System Driver Resident Bytes

9. Virtual Memory Manager • Memory allocation Counters • Available Bytes • Available KBytes • Available MBytes • Pool non-paged Bytes, • Pool Paged Resident Bytes, • System Cache Resident Bytes, • System Code Resident Bytes, • System Driver Resident Bytes

10. What is the difference between Pool Paged Bytes and Pool Nonpaged Bytes? • Which applications would use which VM pool?

11. Virtual Memory map

12. Virtual Memory map • Paged • Memory mapped files • Shared DLLs • File Server • IIS (html, jpg, gif) • Page tables • nonPaged: • I/O buffers used by device drivers • TCP Session data

13. Virtual Memory map • Virtual memory map is static, created once during OS initialization • When any one area is full, memory allocations to that pool will fail • One reserved area of virtual memory can fill up faster than the others;

14. Nonpaged pool size (ref. Q126402): MinimumNonPagedPoolSize = 256KMinAdditionNonPagedPoolPerMb = 32KDefaultMaximumNonPagedPool = 1 MBMaxAdditionNonPagedPoolPerMb = 400KPTE_PER_PAGE = 1024PAGE_SIZE=4096 NonPagedPoolSize = MinimumNonPagedPoolSize + ((Physical MB - 4) * MinAdditionNonPagedPoolPerMB) If NonPagedPoolSize > 128 MB THEN NonPagedPoolSize = 128 MB

15. Paged pool size: Size = (2 * MaximumNonPagedPoolSize) / PAGESIZE Size = (Size + (PTE_PER_PAGE - 1)) / PTE_PER_PAGE PagedPoolSize = Size * PAGESIZE * PTE_PER_PAGE If PagedPoolSize> 192 MB THEN PagePoolSize= 192 MB Note: PagedPoolSize  sizeof (RAM) for systems < 192 MB

16. Default Paged and Nonpaged pool size can be overridden:

17. Virtual Memory constraints • Virtual Memory constraints in 32-bit Windows tend to appear when sizeof (RAM)  4 GB • 2 GB private area is not enough virtual memory for some applications • e.g., SQL Server, Exchange database (store.exe) • Due to fragmentation, it is typically not possible to allocate all 2 GB • 2 GB system area is not enough virtual memory for some applications • File cache for a conventional IIS-managed web site with many static .htm, jpg, gif, etc., files to retrieve

18. Virtual Memory constraints • Virtual Memory constraints in 32-bit Windows tend to appear when sizeof (RAM)  4 GB • Ample RAM exists, but it is not possible for your applications to access it due to virtual memory addressing limitations • Large number of Available Bytes

19. Virtual Memory constraints • So try the /3 GB switch • 1 GB system area is not enough virtual memory for some applications • Possible shortage of Free System Page Table Entries • Possible shortage of Nonpaged Pool • Where Session data from TCP connections is stored • Due to fragmentation, it may not be possible to failover a 2 GB+ private address space (e.g., SQL Server, MS Exchange database – store.exe) using Microsoft Cluster Server (MCS) • During address space recovery on the standby node, the entire virtual memory allocation is acquired at one time

20. Exchange 2000 memory tuning • Exchange default memory allocation parameters are “self-tuning,” but may not be optimal on servers with > 1 GB RAM • Adjust HKLM\SYSTEM\CurrentControlSet\Services\SMTPSVC\Queuing • MsgHandleThreshold & MsgHandleAsyncThreshold HKLM\SYSTEM\CurrentControlSet\Services\Inetinfo\Parameters • FileCacheMaxHandles

21. Exchange 2000 memory tuning • Store DB cache • Store cache normally expands to fill RAM • But, maximum cache = 900 MB • This value can be adjusted using ADSI Edit tool: • msExchESEParamCacheSizeMax • msExchESEParamCacheSizeMin • Also, consider adjusting msExchESEParamLogBuffers attribute for active, back-end servers

22. Detecting memory leaks • Processes that allocate virtual memory, but later “forget” to free it. • MS says leaks won’t happen in .Net managed code due to automatic garbage collection • But, meanwhile…, • Where to look depends on whether process or system addresses are being allocated: • Per Process: Virtual Bytes, Private Bytes, Pool Paged Bytes, Handle Count • System level: Memory Pool Paged Bytes, Pool Nonpaged Bytes and the Objects Object

23. Detecting memory leaks • Look for a steady increase or a sharp spike in process Virtual Bytes, or the System’s Pool Paged Bytes. • If RAM is not full, the leak may also be manifest in the Memory allocation counters and result in increased paging, if RAM fills up. • For example:

24. Detecting memory leaks (1)

25. Detecting memory leaks (2)

26. Detecting memory leaks (2)

27. Detecting memory leaks (2)

28. Detecting memory leaks (2)

29. Lab exercise. • Open SUPPORT.200205080300.EXCHG2K.sum.smf,SUPPORT.200205090300.EXCHG2K.sum.smf, andSUPPORT.200205100300.EXCHG2K.sum.smf and investigate virtual memory usage. • Can you find the application that is leaking?

30. Detecting Memory Leaks (3)

31. Detecting Memory Leaks (3)

32. Detecting Memory Leaks (3)

33. Detecting Memory Leaks (3)

34. Page replacement • Theoretically, Page Faults/sec » Cache faults/sec + Transition faults/sec + Demand Zero Faults/sec + Pages Read/sec • But empirically…

36. Page replacement • Empirically, Page Faults/sec » Transition faults/sec + Demand Zero Faults/sec + Pages Read/sec

37. Questions ?