1 / 59

ADM392 Windows® Server™ 2003 and Windows XP Kernel Changes

ADM392 Windows® Server™ 2003 and Windows XP Kernel Changes. Mark Russinovich Winternals Software. David Solomon Expert Seminars. Outline. Overview Performance Scalability 64-bit support File systems Reliability and recovery Miscellaneous. About The Speakers. Authors of:

sela
Télécharger la présentation

ADM392 Windows® Server™ 2003 and Windows XP Kernel Changes

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ADM392Windows® Server™ 2003 and Windows XP Kernel Changes Mark RussinovichWinternals Software David Solomon Expert Seminars

  2. Outline • Overview • Performance • Scalability • 64-bit support • File systems • Reliability and recovery • Miscellaneous

  3. About The Speakers • Authors of: • Inside Windows 2000, 3rd Edition(Microsoft Press) • Inside Windows 2000/XP/2003 Interactive Internals Video Tutorial • Used by Microsoft for worldwide internal training • David Solomon: • Teaches Windows internals classes (www.solsem.com) • Writes books and articles on Windows internals • Mark Russinovich: • Author of tools on www.sysinternals.com • Co-founder and Chief Software Architect for Winternals Software (www.winternals.com) • Teaches Windows internals classes • Writes books and articles on Windows internals

  4. Level Of Kernel Change • Windows Server 2003 & Windows XP are modest upgrades as compared to the changes from Windows NT 4.0 to Windows 2000 • Kernel architecture is basically unchanged • No new subsystems • No new API sets • Internal version numbers confirm this • Windows 2000 was 5.0 • Windows XP is 5.1 (not 6.0) • Windows Server 2003 is 5.2 • Not the same kernel as XP (a superset) • But, nonetheless, still lots of interesting kernel changes…

  5. Outline • Overview • Performance • Scalability • 64-bit support • File systems • Reliability and recovery • Miscellaneous

  6. The Boot Process • Goal: From power on to logon screen in under 30 seconds • Boot monitoring tool (Bootvis) developed to help Microsoft and hardware vendors optimize • Prefetching of drivers • I/O overlapped with device initialization • Slow drivers do work asynchronously • Winlogon doesn’t wait for Workstation service to start if • Account doesn't depend on a roaming profile • Domain policy that affects logon hasn't changed since last logon

  7. Prefetch Mechanism • File activity is traced and used to prefetch data the next time • On boot, system monitors first 2 minutes of boot process (stops 30 seconds after the user starts the shell or 60 seconds after all services are started) • Also applies to application startup • First 10 seconds are monitored • Prefetch “trace file” stored in \Window\Prefetch • Name of .EXE-<hash of full path>.pf • Boot trace: NTOSBOOT-B00DFAAD.pf

  8. Prefetch Mechanism • When application run again, system automatically • Reads in directories referenced • Reads in code and file data • Reads are asynchronous • But waits for all prefetch to complete • In addition, every 3 days, system automatically defrags files involved in each application startup! • Bottom line: Reduces disk head seeks • This was seen to be the major factor in slow application/system startup

  9. Hibernate And Resume • Hibernation file is better compressed • I/O overlapped on IDE drives • Resume is faster • Reads are larger • Device parallelization during power up improved • Power up done asynchronously in the background by drivers (specifically power-pagable devices without children)

  10. Other Performance Improvements • Fast system calls • Uses SYSENTER/SYSEXIT on Pentium II or higher; SYSCALL on AMD • More intelligent working set trimming on MP systems • Pages removed are LRA (Least Recently Accessed) • In Windows 2000, was only done on uniprocessor systems

  11. Outline • Overview • Performance • Scalability • 64-bit Support • File systems • Reliability and Recovery • Miscellaneous

  12. SMP Scalability • Scalability improvements made in several areas of the kernel • Some of these are in Windows XP • More are in Server 2003 • Several areas: • Increased physical memory support • Bigger multiprocessor systems • Improved synchronization • New types of multiprocessor systems • Increases in system virtual memory limits

  13. Physical Memory Limits • 32-bit Server 2003 Enterprise Edition supports 32 GB RAM • Windows 2000 Advanced Server limit was 8 GB • 32-bit Server 2003 Datacenter Edition supports 128 GB • Windows 2000 Datacenter Server was 64 GB • 64-bit Sever 2003 Datacenter supports 512GB (!)

  14. Using Extended Physical Memory • On 32-bit Windows, virtual address space is still 4 GB, so how can you “use” > 4 GB of memory? 1. Although each process can only address 2 GB (or 3 GB), many may be in memory at the same time (e.g. 5 * 2 GB processes = 10 GB RAM used) 2. Files in system cache remain in physical memory • Although file cache doesn’t know it, memory manager keeps unmapped data in physical memory 3. Address Windowing Extensions allow Win32 processes to allocate more than 2 GB of memory • Map windows as needed

  15. Large Pages • Large pages allow a single page directory entry to map a larger region • x86: 4 MB • Itanium: 16 MB • Large pages are used to map NTOSKRNL, HAL, boot drivers, and nonpaged pool if a “large memory system” • Windows 2000: 128 MB or more • Windows XP/2003: 256 MB or more • Advantage: improves performance • Single TLB entry used to map larger area • New in Server 2003: applications can VirtualAlloc large pages with MEM_LARGE_PAGE flag

  16. Large Pages • Disadvantage: disables kernel write protection • With small pages, OS/driver code pages are mapped as read only; with large pages, entire area must be mapped read/write • Drivers can then modify/corrupt system & driver code without immediately crashing system • Can override by changing HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management • LargePageMinimum REG_DWORD -1 • EnforceWriteProtection REG_DWORD 1

  17. Larger Multiprocessor Systems • 64-bit Windows Server 2003, Datacenter edition supports 64 CPUs

  18. SMP Scalability • New, more efficient locking mechanism (pushlocks) • Doesn’t use spinlocks when no contention • Used for object manager and address windowing extensions (AWE) related locks • Minimized lock contention for hot locks • E.g., PFN (Page Frame Database) lock • Some locks completely eliminated • Charging nonpaged/paged pool quotas, allocating and mapping system page table entries, charging commitment of pages, allocating/mapping physical memory through AWE functions

  19. Per-CPU Scheduling Queues • Before, there was one system-wide list of threads that want to run • System had to lock this database to decide which thread to run next • Now, each CPU has its own list of threads that want to run • Threads always go into the ready queue of their ideal processor • Instead of locking the dispatcher database to look for a candidate to run, per-CPU ready queue is checked first • If there is one, does context swap • Else scans other CPU’s ready queues looking for a thread to run • This scan is done OUTSIDE the dispatcher lock • Just acquires per-CPU scheduling database lock • Global dispatcher lock still acquired to wait or unwait a thread and/or change state of a dispatcher object • Bottom line: dispatcher lock is now held for a MUCH shorter time

  20. Hyperthreading • Support for logical processors on hyperthreaded Xeon & Pentium 4 processors • Does not count logical processors against CPU license limit like Windows 2000 • E.g., Windows Server 2003 Enterprise Edition will use 16 logical processors on an 8 way hyperthreaded Xeon system • Windows 2000 Advanced Server would only use 8 • Scheduling algorithms take into account logical vs physical processors • Used in choosing idle CPU to run a thread

  21. NUMA • NUMA (non uniform memory architecture) systems • Groups of physical processors (called “nodes”) that have local memory • Connected to the larger system through a cache-coherent interconnect bus • Still an SMP system (e.g. any processor can access all of memory) • But node-local memory is faster • Scheduling algorithms take this into account • Tries to schedule threads on processors within the same node • Tries to allocate memory from local memory for processes with threads on the node • New Win32 APIs to allow applications to optimize

  22. System Virtual Memory Limits • Key system memory limits raised in XP & Server 2003 • Windows 2000 limit of 200 GB of mapped file data eliminated • Previously limited size of files that could be backed up • Maximum System Page Table Entries (PTEs) increased • Can now describe 1.3 GB of system space (960 MB contiguous) • Windows 2000 limit was 660 MB (220 MB contiguous) • Increases number of users on Terminal Servers • Also means maximum device driver size is now 960 MB (was 220 MB)

  23. Registry Limits • SYSTEM hive was limited to 12MB in Windows 2000 • Now limited to 200 MB or ¼ of RAM, whichever is lower • Total loaded registry hive data was limited to 376MB in Windows 2000 • Limited number of terminal server users • This was because registry hives were read into paged pool when loaded • Explains why there was a system registry quota • XP/2003: No limit to loaded registry hive data • Registry no longer in paged pool • Hives are accessed as memory mapped files • Views are mapped as necessary

  24. Outline • Overview • Performance • Scalability • 64-bit support • File systems • Reliability and recovery • Miscellaneous a

  25. Windows 64-Bit Editions • Supports 64-bit Itanium Intel architecture • 64-bit Edition 2003 will support AMD Opteron and Athlon 64 • Products • Windows XP Professional 64-bit edition • Windows Server 2003 64-bit editions • True 64-bit versions (e.g. pointers are 64-bits) • Much larger address space • Good for CAD, simulation, other memory-intensive applications • Not a performance boost in and of itself

  26. Itanium Address Space Layout 0 User-Mode User Space 6FC00000000 Kernel-Mode User Space User Page Tables 1FFFFF0000000000 Session Space 2000000000000000 Session Space Page Tables 3FFFFF0000000000 System Space E000000000000000 -E000060000000000 Session Space Page Tables FFFFFF0000000000 64-bit Windows 32-bit Windows User Address Space 7152 GB (6.9 TB) 2 or 3 GBSystem PTEs 128 GB 1.3 GBSystem cache 1024 GB (1 TB) 960 MBPaged pool 128 GB 470 MBNon-paged pool 128 GB 256 MBPage file size 32 TB 16 TB

  27. 32-Bit Application Support • “Wow64” - allows execution of Win32 32-bit applications on 64-bit OS • Wow64.dll - provides core emulation infrastructure and thunks for Ntoskrnl.exe entry-point functions • Loads the x86 version of Ntdll.dll and runs its initialization code, which loads all necessary 32-bit DLLs • 32-bit Kernel32.dll, ntdll.dll, etc., are loaded from %systemroot%\SysWOW64 • Wow64win.dll - provides thunks for Win32k.sys entry-point functions • Wow64cpu.dll - provides x86 instruction emulation; executes mode-switch instructions on Itanium

  28. Wow64 • Some advanced Win32 APIs not supported (e.g. scatter/gather I/o) • Interoperability • COM, cut/paste interoperate • Cannot load 32-bit DLLs in 64-bit process and vice versa • On Itanium, slower execution than on native 32-bit machine • Images marked large address space aware get a full 4 GB process virtual address space • OS isn’t mapped there, so space is available for process

  29. Win64 Disk Partitioning • Win64 boot.ini is in non-volatile RAM • Extensible Firmware Interface (EFI) • First partition is FAT • GUID Partition Table (GPT) • 64-bit only • Overcomes limitations of MBR partitioning • 64-bit offsets and lengths • Partition table is mirrored • No nesting

  30. Outline • Overview • Performance • Scalability • 64-bit support • File systems • Reliability and recovery • Miscellaneous

  31. File System Enhancements • FAT32 on DVD-RAM • Read-only NTFS volumes • UDF 2.01 (new standard for DVD-ROM, DVD-RAM, DVD-RW, DVD video) • Encrypting File System (EFS) • No longer a separate driver—integrated into NTFS • Supports multi-user access to encrypted files (supports file sharing)

  32. The Defrag API • Completely rewritten API • Can defrag MFT and other metadata files (except log file, paging file) • Can defrag encrypted files • No 4KB-cluster limit on NTFS • Command line interface (scriptable)

  33. Volume Shadow Copy • Volumes can be “snapshotted” • Allows “hot backup” (including open files) • Uses copy on write • Changes to volume after snapshot cause original contents of cluster to be stored in snapshot file • Later, reads to changed data return contents at time of snapshot • Applications can tie in with mechanism to ensure consistent snapshots

  34. Volume Snapshots Writers Backup Application Oracle 5. Backup applicationsaves data from volume Shadow copies 2. Writers told to freeze activity • Backup application requests shadow copy Volume Shadow Copy Service SQL 4. Writers told to resume (“thaw”) activity 3. Providers asked to create volume shadow copies Volume Shadow Copy Driver (volsnap.sys) Mirror provider Providers

  35. Shadow Copies of Shared Folders • When enabled, 2003 Server uses shadow copy to periodically create snapshots of volumes • Schedule and space used is configurable

  36. Shadow Copies on Shared Folders • Shadow copies only exposed as network shares • Clients install Explorer extension that integrates with server that let’s them • View the state of folders and files within a snapshot • Rollback individual folders and files to a snapshot

  37. Outline • Overview • Performance • Scalability • 64-bit support • File systems • Reliability and recovery • Miscellaneous

  38. System Restore • Rollback system to previous state: • Registry, COM+ registration database, user profiles, other files not protected by WFP • Windows XP only (not on Server) • Replacement of certain file types causes original version to be stored in a restore point folder • 569 file types monitored – see Platform SDK for list • Restore operation replaces these files • Implemented as a service and a filter driver

  39. System Restore Applications User mode Kernel mode File system request System Restore Filter Change.log1 File System Driver (NTFS/FAT) A0009653.exe A0009654.ini \System Volume Information\ _restore{XX-XXX-XXX }\ RP5

  40. System Restore • Restore Points are created • Every 24 hours • When installing an unsigned driver • When explicitly requested by user or an install program (via an API or script) • WMI interfaces allow scriptable control • Create/delete restore points, change configuration

  41. Driver Rollback • System saves updated driver in \Windows\System32\ReinstallBackups\nnnn\DriverFiles • New button on device properties to roll back driver • If you choose roll back, also saves a copy in \Windows\LastGood \System32\Drivers • Will then automatically roll back driver when booting from “last known good”

  42. Driver Verifier Enhancements • New verification options: • DMA verification – detects improper use of DMA buffers, adapters, and map registers • Deadlock detection – detects lock hierarchy violations with spinlocks, mutexes, fast mutexes • SCSI verification - monitors the interaction between a SCSI miniport driver and the port driver • Enhanced I/O Verification tests drivers' support for power management, WMI, and filters • Simpler wizard-style GUI (verifier.exe) • Defaults verify unsigned drivers

  43. Side-By-Side Assemblies • Microsoft wants to end DLL hell by letting applications specify DLLs they use by version • Support multiple versions simultaneously installed • Application will use updates only if backward compatible • Application that uses assemblies has a manifest file • XML file that specifies application version number and DLLs • DLLs are identified by GUIDs and version number and are stored either in the application’s directory or in SystemRoot\Winsxs

  44. Theme-Aware Common Controls • Example: Windows XP Common Control DLL (comctrl32.dll) • Windows XP version is 6, which supports Luna themes • Windows 2000 version is 5, which doesn’t support themes • Non-theme aware applications can behave incorrectly if used with v6 controls, • If an application doesn’t have a manifest that specifies v6, it gets v5, which is in the SystemRoot\System32 directory

  45. Outline • Overview • Performance • Scalability • 64-bit support • File systems • Reliability and recovery • Miscellaneous

  46. Miscellaneous • Boot and execute from ROM • OS and drivers copied to RAM • Applications can execute from ROM • Hot plug memory • Hot plug PCI • Headless server support (no keyboard, video, mouse) • Remote Installation Service • EMS (Emergency Management Service) allows remote disaster recovery/control via serial port or network

  47. Terminal Services • Terminal Services included with Windows XP supports multiple sessions • Home Edition: Supports “disconnect and switch users” • Professional: Remote Desktop Connection • Remote desktop redirection for audio, serial/parallel port, file system (local drives) • Server 2003: Load balancing support, remote audio, local drive & printer mapping

  48. Services Infrastructure • More services run in generic service host process (svchost.exe) • Reduces number of processes • Two new less privileged accounts for built-in services • LOCAL SERVICE, NETWORK SERVICE • Less rights than SYSTEM • Reduces possibility of damage if system compromised • Four instances of Svchost (at least) • SYSTEM • SYSTEM (2nd instance – for RPC) • LOCAL SERVICE • NETWORK SERVICE

  49. Debugging • Can now detach debugger without killing debuggee • See new Win32 DebugActiveProcessStop • Kernel debugging • Live local system kernel debugging (kd –kl or windbg –kl) • Kernel debugging over 1394 (in addition to serial) • Auto load of updated drivers to target

  50. Registry Callbacks • Up until now Regmon has relied on system call “hooking” to intercept Registry accesses • Hooking isn’t supported by the kernel • As of XP the system call table is write-protected by default if a system has < 256 MB, requiring a trick • Server 2003 introduces a Registry callback mechanism • Driver can see and modify Registry behavior • Latest version of Regmon comes with two drivers: one for Server 2003 and one for previous versions

More Related