VIR321 Advanced Storage Infrastructure Best Practices to Enable Ultimate Hyper-V Scalability

2. VIR321Advanced Storage Infrastructure Best Practices to Enable Ultimate Hyper-V Scalability Txomin Barturen Senior Manager, Partner Engineering EMC Corporation

3. Cloud Computing "Enterprise cloud" implies a further promise of rigorous and audited security, high availability and robust capabilities for customization and integration. - Peter Coffee, SalesForce.COM Virtualized Customizable Implement a Customizable,DynamicVirtualizedenvironment That is both Scalableand Highly Available Dynamic Highly Available Scalable

4. Agenda • Hyper-V as a platform • Choices for provisioning • Challenges for administration • Cluster Shared Volumes • Deployment Methodologies • Host based mechanisms • Storage based deployment • Maintaining availability • Windows Failover Clustering : Cluster Shared Volumes

5. Microsoft Hyper-V • Microsoft’s resource light Hypervisor • Enables VM consolidation at scale • Parent operating system providesresource scheduling system • Child virtual machines accessvirtualized resources and services • CPU, Memory, Network, Storage • Maximum number of virtual machinesbased on physical resources of parent • Memory and CPU

6. Management of Hyper-V Environments • Local mode • Single Hyper-V server • Tool: Microsoft Hyper-V Manager • Highly Available mode • Up to 16 nodes in Windows Failover Cluster • Tool: Windows Failover Cluster Manager • Built-in support for Hyper-V management • Enterprise mode • System Center Virtual Machine Manager • Multiple server instances • Includes support for Clustered instances • Includes support for VMware

7. Deployment Models for Virtual Machines SCVMM Server • Local mode • Define VM • Boot ISO and install • Export/Import • Highly Available mode • Same options as Local mode • Enterprise mode • SCVMM • Library methodology • Usage of templates • Customized images are sysprep-ed and stored • Template can then be used to mass create virtual machines • Each is individualized during deployment Templates

8. Storage Choices for Virtual Machines • Virtual Machine Storage 1. As a Virtual Hard Drive (VHD) • Fixed size VHD • Dynamic VHD • ~2TB max size 2. As a Pass Thru Device • No size limit, just NTFS 3. As an iSCSI device to the VM • Guest Cluster • Cluster Shared Volumes • Centralized shared storage for VHDs 1 VHD 3 VHD Parent managed LUN (FC or iSCSI) iSCSI via network 2 Disk offline to Parent

9. Cluster Shared Volumes: Availability Built-in • Feature of Windows Server 2008 R2 Failover Clustering • Single namespace for LUNs • All volumes directly accessible by each physical server • One node acts as coordinator for locking • Locking limited to individual files on create / open, so overhead is small • VMs are no longer tied to the LUN that supports them • No failover requirement for LUN as it is fully online C:\ClusterStorage\Volume1 C:\ClusterStorage\Volume2 C:\ClusterStorage\Volume3 C:\ClusterStorage\Volume1 C:\ClusterStorage\Volume2 C:\ClusterStorage\Volume3 C:\ClusterStorage\Volume1 C:\ClusterStorage\Volume2 C:\ClusterStorage\Volume3 C:\ClusterStorage\Volume1 C:\ClusterStorage\Volume2 C:\ClusterStorage\Volume3

10. Failover Clustering: Cluster Shared Storage • Provides two major benefits 1. Mitigates discrete LUN management 2. High Availability Consider the challenge of thousands of VMs with one or more LUNs each • Discrete LUN management • In large scale deploymentsadministration can be a challenge • Windows 256 LUN per target limit • High Availability • All LUNs fully accessible from all nodes • No requirement for ownership transition • Reduced complexity with lesser numberof LUNs to manage for given VM pool A smaller number of LUNs with multiple VHDs – all fully accessible VHDs VHDs VHDs • But what about performance?

11. Storage Provisioning Can Be Challenging • Performance based sizing can required per-spindle calculations • Where the component are defined as: • PhysicalIO = Actual back-end I/O against the physical spindles • Total_IO = Host generated I/O workload • Read_IO% = percentage of Total_IO that is a read workload • Read_Hit = amount of read workload services from array cache • Write_IO% = percentage of Total_IO that is a write workload • RaidFactor= Additional I/O activity required for write operations to cater for parity calculations ( 2 for RAID 1, 4 for RAID 5, 6 for RAID 6) • Spindle count rule of thumb for 8 KB I/O size (SQLCAT) • 10K RPM – 100-130 IOPS at ‘full stroke’ 15K RPM – 150-180 IOPS at ‘full stroke’

12. Storage Allocation : Efficiency and Dynamic VHDs • Dynamic VHDs • Mitigate storage allocation at onset • Allow for dynamic growth on an as-needed basis • Challenge: Inefficient I/O profile • Misalignment is inherent with Dynamic VHDs – meta-data components • At low I/O rates this may be of no concern • At high I/O rates (consolidation) this should be avoided • I/O misalignment : the issue RAID penalties for writes also apply and are doubled! R1 : 4 X R5 : 8 X R6 : 12 X 8 KB 8 KB Host Disk

13. Why Dynamic VHDs? • Address storage allocation – Avoid full allocation at onset • Allow for dynamic growth • Could storage mechanisms fulfill the requirement? • Thin devices would represent a similar function • EMC refers to these as Virtually Provisioned devices • Includes both a space saving and performance function

14. Virtual Provisioning : Simplified Performance Design • Reduce storage provisioning overhead • Virtual Machines rarely fully utilize the storage allocated to them • Allocate on-demand is more cost effective • Applications running within the virtual machine can be “thin” efficient • 90% of storage consumption comes from the storage provisioned for the applications Virtual Provisioning: allows you to save space and service performance Configured asRAID-5 3+1 Bound toThin Pool Allocations occur in extents, across all available data devices and the disks that they are defined on. Traditional StorageDevice Thin StorageDevice Performance derived from all spindles with data devices (RAID protected). Support for thousands of spindles. Performance derived from four physical spindles. Potentially shared with other devices.

15. Scalability for SnapShots • Array based SnapShots can provide a scalable solution for “Templates” • Efficiencies are possible utilizing SnapShots to cater for duplication • Only changes need to be stored • Majority of “Gold” image may never change across VMs • Some SnapShot solutions provide a pool mechanism to cater for storage and deal with performance Source Volume Allocations occur across all available save devices and the disks that they are defined on. Snap Volume Some arrays provide upwards of 128 SnapShot session Performance derived from all spindles with save devices (RAID protected). Support for thousands of spindles.

16. Unlocking Performance: Many Layers • When consolidating workloads for co-located virtual machine VHDs • LUN must support the I/O load • Workload for CSVs can come from all nodes • Entire I/O stack must support the aggregate workload • Host HBA connectivity must be considered • SAN Fabric should be scalable and redundant • Array connectivity must also scale FA HBAs Fabric FA Disk Resources HBAs Array Ports

17. Host-level Scalability with Windows MPIO • Multi-pathing from Parent for best performance • Vendor specific MPIO DSM may provide array-specific optimizations • Recommend around 4 discrete paths

18. Building a Large Scale Deployment • Unsurprisingly using a Symmetrix VMAX storage array • Utilizing Virtually Provisioned storage devices • SnapShot technology for creating replicas • 16 Nodes Windows Server 2008 R2 Failover Cluster • 64 Virtual Machines per node • Target at least 100,000 IOPs for environment • Utilize Cluster Shared Volumes

19. Environment Overview 16 FC connections Dual connectionsper server 8 Gb Dual FabricSAN SCVMM Server • Network Infrastructure • 3 x 1 Gbit Ethernet • 1 Management HBA • 2 Deployment / Live Migration HBA All nodes useBoot from SAN

20. Initial Deployment Methodology - SCVMM • Using SCVMM and Templates – Network based copy VHDs are around 50 GB in size Network can quickly become the bottleneck – mechanism is reliable

21. Automating SCVMM Deployment Model • PowerShell based mechanism • Easily derived from SCVMM deployment script • With a modest amount of customization • Unfortunately for 1024 VMs (16 nodes with 64 VMs each) this approach would take 52 hours

22. Storage Based Implementation VMs customizedas necessary 1. Create a Master LUN 2. Generate sysprep-ed VHDs • Use ExportVirtualSystemEx.vbs 3. Replicate VHDs as necessary 4. Duplicate Master Image 5. Attach to nodes andinstantiate virtual machines 2 1 3 4 Replicas are SnapShots andconsume no space when created.

23. Sample of Disk Import PowerShell 2. Re-signaturing Disks if ($DiskTyp -eq "MBR") { # We need to zap the signature, and Windows will assign a new one. symntctl signature -sid $MySID -symdev $MyDev -erase if ($NewSIU = symntctl show -sid $MySID -symdev $MyDev) { if (($LastExitCode) -or ($SIU -eq "") -or ($SIU.Length -lt 9)) { Write-Error "Processing of device signature failed" Exit 1 } else { $NewSig = $NewSIU[4].Substring(13,8) $DiskId = "0x" + $NewSig } } 1. Importing Disks if ($SIU = symntctl show -sid $MySID -symdev $MyDev) { if (($LastExitCode) -or ($SIU -eq "") -or ($SIU.Length -lt 9)) { … Error check – removed for readability } } else { … Error check – removed for readability } $SelDisk = $SIU[1].Substring(19,2) $DiskTyp = $SIU[2].Substring(18,3) $OrigSig = $SIU[4].Substring(13,8) # We process the disk via DISKPART $command= @" select disk $SelDisk online disk noerr attr disk clear readonlynoerr "@ $command| DiskPart 3. Adding as CSVs $NewDisk = Get-ClusterAvailableDisk | ?{ $_.Id -eq $DiskId } | Add-ClusterDisk $MyCSV = Add-ClusterSharedVolume $NewDisk.Name

24. Comparison of Deployment Models in Action demo

25. Comparison Results • SCVMM Model • Deployments limited by network • More scalability by adding H/W or using 10 Gbit Ethernet • More flexible • Projected: 1024 VMs in 52 hrs • Storage Array Model • Use H/W assisted replication • Limited in terms of replication offerings • Possible limits in number of replicas • Symmetrix VMAX support 128 Snaps per source LUN • Better for well structured configurations • 1024 VMs in ~ 1.5 hrs (depending on CPU load of servers)

26. Cluster Shared Volumes and Scalable I/O • How much can a CSV handle? • It’s actually more about what the storage design can deliver

27. Performance Results

28. Performance – Staging all 1024 VMs

29. 2 x 8 TB LUN per cluster Virtual Provisioning – 400 drives 300+ Virtual Machines 50 GB Boot VHD Data VHDs as needed SQL Srv. include multiple VHDs What Customers Build 7 x 16-Node Clusters 8 TB LUN 8 TB LUN

30. Summary • Scalability is available at all levels of solution design • Various alternatives in deployment models • We only investigated a small number of all offerings • The “Right Choice” will depend on the specific scenario • A blend of all the choices available may be “Right” solution • Large scale systems resulting from consolidation require • Scalable architecture • Hardware can assist • High Availability options • Failover Clustering and CSVs can provide an answer Implement a Customizable,DynamicVirtualizedenvironment That is both ScalableandHighly Available

32. Value Add : Providing D/R to the Environment Cascaded Replication Concurrent Replication Heterogeneous Replication

33. Session Evaluations Tell us what you think, and you could win! All evaluations submitted are automatically entered into a daily prize draw*  Sign-in to the Schedule Builder at http://europe.msteched.com/topic/list/ * Details of prize draw rules can be obtained from the Information Desk.

34. © 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.