Storport Smorgasbord A selection of today’s hottest Storport topics

2. Storport Smorgasbord A selection of today’s hottest Storport topics • Keith Frechette • Software Development Engineer • Device & Storage Technologies

3. Agenda • Half-duplex Mode: Not appropriate for shipped products • StorPortCompleteRequest: Fouled-up beyond all repair? • VPD Page 0x83 Data: Storport changes that you should test now • New Storport event log interface • NUMA and multiprocessor improvements • Performance Tip: Completing requests during StartIo • New Storport miniport sample • Virtual Miniports: When are they appropriate?

4. Half-duplex Mode: Not appropriate for shipped products

5. Don’t Ship Them! • Half-duplex mode: • Is intended only for short-term use during the initial porting of a miniport from SCSIport to Storport • Restricts port/miniport synchronization to that of SCSIportminiports • Results in more time spent at device IRQL • Reduces concurrency in I/O processing, leading to lower performance • Is not compatible with new Storport interfaces and optimizations • Is not appropriate for use in shipping products • If you continue to ship a half-duplex-based miniport, you risk compatibility problems with future Storport revisions.

6. StorPortCompleteRequest: Fouled-up beyond all repair?

7. Background: How did it ever work? • The design and implementation were inherited from SCSIport • Intended as a convenience function for simplified reset recovery • Works well with non-reentrant model of SCSIport SCSIport Miniport calls ScsiPortCompleteRequest to fail all outstanding requests Miniport SCSIport detects an I/O timeout and issues a reset Meanwhile, a new request is received They are sent to the miniport ScsiPortCompleteRequest completes I/O requests arrive Miniport clears its queue I/O resumes Recovery complete! I/O Queue Read: sectors 10-15 Read: sectors 10-15 Read: sectors 10-15 Read: sectors 2003-2008 Read: sectors 1430-1455 Read: sectors 1430-1455 Write: sector 11 Write: sector 11 “Outstanding” Requests Read: sectors 2003-2008 ScsiPortCompleteRequest

8. The Problem • Storport miniports are reentrant, so StorPortCompleteRequest leads to: • Confusion over which requests were completed and which weren’t • “Double completions” and “never completions” Storport Miniport calls StorPortCompleteRequest to fail all outstanding requests Miniport Storport detects an I/O timeout and issues a reset Meanwhile, a new request is received Uh-oh.  StorPortCompleteRequest completes Miniport clears its queue I/O Queue Read: sectors 10-15 Read: sectors 10-15 Read: sectors 10-15 Read: sectors 2003-2008 Read: sectors 1430-1455 Read: sectors 1430-1455 Write: sector 11 Write: sector 11 “Outstanding” Requests Read: sectors 2003-2008 StorPortCompleteRequest

9. The Recommendation • The miniport individually flushes the outstanding requests • Pause the Storport I/O queue • Call StorPortNotification( RequestComplete ) for each outstanding request • Resume the Storport I/O queue Storport Miniport Storport detects an I/O timeout and issues a reset Miniport calls StorPortNotification (RequestComplete) for each request Miniport calls StorPortPauseDevice Meanwhile, a new request is received Miniport calls StorPortResumeDevice Miniport clears its queue I/O resumes Recovery complete! I/O Queue Read: sectors 10-15 Read: sectors 10-15 Read: sectors 2003-2008 Read: sectors 1430-1455 StorPortPauseDevice StorPortResumeDevice Write: sector 11 “Outstanding” Requests Read: sectors 10-15 Read: sectors 2003-2008 StorPortNotification (RequestComplete) Read: sectors 1430-1455 Write: sector 11

10. The Plan • At present, StorPortCompleteRequest should not be used. • The miniport should manually complete all outstanding requests. • StorPortCompleteRequest will remain so that existing miniports continue to function. • Although there is currently no Windows Logo Program requirement, you can expect that we’ll start checking for the use of StorPortCompleteRequest and warning vendors about likely side-effects. • Based on Microsoft development priorities and on IHV feedback, StorPortCompleteRequest may be: • Redesigned to work properly, although miniport changes may be required • Completely replaced with a new interface designed to fit the Storport model • Simply left as-is to let existing miniports continue to run.

11. VPD Page 0x83 Data: Storport changes that you should test now

12. Overview • Storport’s evaluation of vital product data (VPD) Page 0x83 data has been changed with respect to identifying when a LUN has changed. • Why? Because the current implementation: • Gives equal weight to globally-unique and non-globally-unique identifiers • Causes Storport to fail to detect a disk swap when the two otherwise- unique devices share a non-globally-unique identifier. • The new approach gives precedence to globally-unique identifiers. • Applications that relied on the older approach may behave incorrectly.

13. New Handling: Pseudo-Code • Notes • Identifier types • 0h – Vendor specific • 1h – T10 vendor ID based • 2h – EUI-64 based • 3h – NAA • 8h – SCSI name string • Only LUN-associated IDs are compared. boolCompareIdentiferData( CurrentIdSet, PreviousIdSet ) { if (neither ID set has page 0x83 IDs)     if (page 0x80 exists for both ID sets and is identical)         return Match;     else         return NoMatch; if (only one of the ID sets has page 0x83 IDs)     return NoMatch; if (both ID sets have type 2/3/8 IDs)     if (any IDs of type 2/3/8 match)         return Match;     else         return NoMatch; if (both ID sets have type 1 IDs)     if (any IDs of type 1 match)         return Match;     else         return NoMatch; if (both ID sets have type 0 IDs)     if (any IDs of type 0 match)         return Match;     else         return NoMatch; return NoMatch; }

14. Test It with Your Products! • You should test your storage configurations now and provide feedback. • These changes are very significant and will impact some scenarios. • Example: A firmware update that replaces all the globally-unique IDs but leaves an existing non-globally-unique ID. • With old VPD page 0x83 handling, Storport does not detect a disk swap. • With new VPD page 0x83 handling, Storport detects a disk swap.

15. New Storport event log interface

16. Overview • Alternative to using StorPortLogError • Provides same capabilities as IoWriteErrorLogEntry • Miniport-specific event codes and strings • Miniport-specific “dump data” • Callable at IRQL <= DISPATCH_LEVEL • Includes Storport-specific information • Path/Target/LUN IDs • Storport-specific event codes and strings • Miniport decides which device object to associate • Adapter device object for adapter- or target- associated events. • LUN device object for LUN-associated events. • Implemented as a Storport extended function • Enables backward compatibility with older versions of Storport

17. Usage (At a Glance) STOR_LOG_EVENT_DETAILS logEvent; PWSTR logStrings[3]; . . . logEvent.PathId = path; logEvent.TargetId = target; logEvent.LunId = lun; logEvent.EventAssociation = StorEventLunAssociation; logEvent.StorportSpecificErrorCode = FALSE; logEvent.ErrorCode = MY_MINIPORT_CUSTOM_STATUS_CODE; logEvent.DumpDataSize = sizeof(MY_MINIPORT_EVENT_DATA); logEvent.DumpData = &MyEventData; logEvent.StringCount = ARRAYSIZE(logStrings); logEvent.StringList = logStrings; . . . StorPortLogSystemEvent( pMyDeviceExtension, &logEvent, NULL );

18. Interface Description: Function Interface StorPortLogSystemEvent Format: StorPortLogSystemEvent( HardwareExtension, LogDetails, MaximumSize ) PVOID HardwareExtension Hardware extension for the adapter. PSTOR_LOG_EVENT_DETAILS LogDetails Information to appear in the system event log entry. int * MaximumSize Optional (can be NULL). Variable to receive the maximum combined size of the miniport’s dump data and strings. Only returned if the function failed with STOR_STATUS_INVALID_BUFFER_SIZE.

19. Interface Description: Event Log Details typedefstruct _STOR_LOG_EVENT_DETAILS { ULONG InterfaceRevision; // Revision of this interface ULONG Size; // Size of this structure ULONG Flags; // Special flags to control this interface STOR_EVENT_ASSOCIATION_ENUM EventAssociation; // See STOR_EVENT_ASSOCIATION_ENUM enumeration ULONG PathId; // Path portion of SCSI address ULONG TargetId; // Target portion of SCSI address ULONG LunId; // LUN portion of SCSI address BOOLEAN StorportSpecificErrorCode; // TRUE if error code is specific to Storport (SP_xxx) ULONG ErrorCode; // Error code, either SP_xxx or standard NTSTATUS code ULONG UniqueId; // Often used to indicate the location of the error ULONG DumpDataSize; // Size in bytes of private data block PVOID DumpData; // Private data block ULONG StringCount; // Count of NUL-terminated Unicode strings PWSTR * StringList; // Unicode strings used for substitution within // log messages (%2, %3, %4...) } STOR_LOG_EVENT_DETAILS, *PSTOR_LOG_EVENT_DETAILS;

20. Example 1: Standard Storport Error Code void LogBadFirmware( SAMPLE_DEVICE_EXTENSION * pDeviceExtension, ULONG CurrentRevision ) { STOR_LOG_EVENT_DETAILS logEvent; SAMPLE_PRIVATE_LOG_DATA privateData; privateData.CurrentRevision = CurrentRevision; memset( &logEvent, 0, sizeof(logEvent) ); logEvent.InterfaceRevision = STOR_CURRENT_LOG_INTERFACE_REVISION; logEvent.Size = sizeof(logEvent); logEvent.EventAssociation = StorEventAdapterAssociation; logEvent.StorportSpecificErrorCode = TRUE; logEvent.ErrorCode = SP_BAD_FW_ERROR; logEvent.DumpDataSize = sizeof(privateData); logEvent.DumpData = &privateData; StorPortLogSystemEvent( pDeviceExtension, &logEvent, NULL ); }

21. Example 2: Custom Error Code and Strings void LogBadFirmware( SAMPLE_DEVICE_EXTENSION * pDeviceExtension, PWSTR pCurrentRevString, PWSTR pMinSupportedRevString ) { STOR_LOG_EVENT_DETAILS logEvent; PWSTR logStrings[2]; logStrings[0] = pCurrentRevString; logStrings[1] = pMinSupportedRevString; memset( &logEvent, 0, sizeof(logEvent) ); logEvent.InterfaceRevision = STOR_CURRENT_LOG_INTERFACE_REVISION; logEvent.Size = sizeof(logEvent); logEvent.EventAssociation = StorEventAdapterAssociation; logEvent.StorportSpecificErrorCode = FALSE; logEvent.ErrorCode = SAMPLE_INVALID_FIRMWARE_ERROR; logEvent.StringCount = ARRAYSIZE(logStrings); logEvent.StringList = logStrings; StorPortLogSystemEvent( pDeviceExtension, &logEvent, NULL ); }

22. NUMA and multiprocessor improvements

23. Summary of New Interfaces New Multiprocessor Interfaces StorPortGetCurrentProcessorNumber StorPortGetGroupAffinity StorPortGetActiveGroupCount New NUMA Interfaces StorPortGetNodeAffinity StorPortGetActiveNodeCount StorPortGetHighestNodeNumber StorPortGetLogicalProcessorRelationship • Related Interfaces for Memory Allocation • StorPortAllocateContiguousMemorySpecifyCacheNode • StorPortFreeContiguousMemorySpecifyCache

24. New Multiprocessor Interfaces Use these functions if your miniport can optimize its performance based on knowledge of the underlying processor topology. ULONG StorPortGetCurrentProcessorNumber( PVOID HardwareExtension, PPROCESSOR_NUMBER ProcNumber ) Description Returns the system-assigned number of the current processor on which the caller is running. ULONG StorPortGetGroupAffinity( PVOID HardwareExtension, USHORT GroupNumber, PKAFFINITY GroupAffinityMask ) Description Constructs a mask of the active processors in the requested group. ULONG StorPortGetActiveGroupCount( PVOID HardwareExtension, PUSHORT NumberOfGroups ) Description Returns the number of groups present in the system. Continued…

25. New Multiprocessor Interfaces …continued ULONG StorPortGetLogicalProcessorRelationship( PVOID HardwareExtension, PPROCESSOR_NUMBER ProcessorNumber, LOGICAL_PROCESSOR_RELATIONSHIP_TYPE RelType, PSYSTEM_LOGICAL_PROCESSOR_INFORMATION_EX Info, PULONG Length ) Description Returns a specified type of relationship information about groups, physical processors, and nodes in the host system.

26. New NUMA Interfaces Use these functions if your miniport can optimize its performance based on knowledge of the underlying NUMA/memory topology. ULONG StorPortGetNodeAffinity( PVOID HardwareExtension, ULONG NodeNumber, PGROUP_AFFINTY NodeAffinityMask ) Description Constructs a mask of the active processors in the requested NUMA node. ULONG StorPortGetActiveNodeCount( PVOID HardwareExtension, PULONG NumberOfNodes ) Description Returns the number of nodes present in the system. ULONG StorPortGetHighestNodeNumber( PVOID HardwareExtension, PULONG HighestNode ) Description Returns the largest possible node number on the system.

27. New Memory Interfaces Use these function to dynamically allocate and free contiguous memory blocks. A preferred NUMA node can be specified. ULONG StorPortAllocateContiguousMemorySpecifyCacheNode( PVOID HardwareExtension, SIZE_T NumberOfBytes, PHYSICAL_ADDRESS LowestAcceptableAddress, PHYSICAL_ADDRESS HighestAcceptableAddress, PHYSICAL_ADDRESS BoundaryAddressMultiple, MEMORY_CACHING_TYPE CacheType, NODE_REQUIREMENT PreferredNode, PVOID * BufferPointer ) Description Allocates a range of physically contiguous, cache-aligned memory. ULONG StorPortFreeContiguousMemorySpecifyCache( PVOID HardwareExtension, PVOID BaseAddress, SIZE_T NumberOfBytes, MEMORY_CACHING_TYPE CacheType ) Description Frees a buffer allocated by StorPortAllocateContiguousMemorySpecifyCacheNode.

28. Performance Tip: Completing requests during StartIo

29. Completing Requests During StartIo Handling • By completing I/O requests in its HwStorStartIo routine, a miniport can: • Spend less time at device IRQL, thus improving system responsiveness • Take advantage of new Storport optimizations that further improve system responsiveness and I/O throughput • To take advantage of the new Storport optimizations, a miniport: • Must enable DPC redirection via StorPortInitializePerfOpts • Must use StorPortNotification(RequestComplete) while in its HwStorStartIo routine to notify Storport of completed I/O requests • Should indicate its intention to do completion-during-StartIo by setting the STOR_PERF_OPTIMIZE_FOR_COMPLETION_DURING_STARTIO flag in its call to StorPortInitializePerfOpts • While in its HwStorStartIo routine, a miniport should check for completed requests after starting the requested I/O operation

30. New Storport miniport sample

31. Sample Overview • Port of LSI production SYM_U3 SCSIport based driver to Storport • Supports LSI Ultra160 Parallel SCSI adapters • Older technology SCSI controller hardware (minimal intelligence) • Very simple custom 8-bit RISC processor, 8K internal RAM (circa 1999) • Hardware and scripts limited to use 256 queue tag values per adapter • Hardware and scripts designed to SCSIport capabilities • Adjustments in miniport match Storport advanced functionality to limited hardware capabilities • Porting required about 3 weeks (develop/test) • Will be the Storport miniport sample in the next version of the WDK

32. Porting Activities • Summary of SCSIport to Storport porting activities • Remove all NextRequest and NextLuRequest calls • Rename all ScsiPortxxx calls to StorPortxxx • Add BuildIo routine (moved about 75% of code from StartIo) • Convert driver to full-duplex operation • Add queue management routines (error handling, adapter restart) • Add LUN and target reset support • Add code to assign queue tags internally (hardware limitation) • Add synchronization routines for bus reset and full adapter restart • Add support for SRB_FUNCTION_POWER (adapter shutdown via SRB) • Add StorPortSetDeviceQueueDepth call—LUN queue depth of 31

33. Virtual Miniports: When are they appropriate?

34. When Are They Appropriate? • A virtual miniport is appropriate when: • It is entirely simulating a device (e.g. a RAM disk) • It has no hardware of its own but is using another device (e.g. a network adapter) as a transport for I/O requests • A virtual miniport is inappropriate when: • It is directly controlling real hardware

35. New Bus Types for Virtual Miniports • BusTypeVirtual • Virtual miniport equivalent of BusTypeUnknown • Use when a more-specific bus type isn’t available for the virtual device • Example: RAM disks that are volatile (aren’t persisted) • BusTypeFileBackedVirtual • Indicates that the virtual device persists its “contents” to a data file • Example: A logical disk whose I/O is redirected to an image file on a physical disk drive

36. Additional Resources • Storage in the WDK Documentation on MSDNhttp://msdn.microsoft.com/en-us/library/bb870491.aspx • Related Sessions

37. Questions?