710 likes | 847 Vues
STORAGE ARCHITECTURE/ GETTING STARTED: SAN SCHOOL 101. Marc Farley President of Building Storage, Inc Author, Building Storage Networks, Inc. Agenda. Lesson 1: Basics of SANs Lesson 2: The I/O path Lesson 3: Storage subsystems Lesson 4: RAID, volume management and virtualization
E N D
STORAGE ARCHITECTURE/GETTING STARTED:SAN SCHOOL 101 Marc Farley President of Building Storage, Inc Author, Building Storage Networks, Inc.
Agenda • Lesson 1: Basics of SANs • Lesson 2: The I/O path • Lesson 3: Storage subsystems • Lesson 4: RAID, volume management and virtualization • Lesson 5: SAN network technology • Lesson 6: File systems
Lesson #1 Basics of storage networking
Connecting • Networking or bus technology • Cables + connectors • System adapters + network device drivers • Network devices such as hubs, switches, routers • Virtual networking • Flow control • Network security
Storing • Device (target) command and control • Drives, subsystems, device emulation • Block storage address space manipulation (partition management) • Mirroring • RAID • Striping • Virtualization • Concatentation
Filing • Namespace presents data to end users and applications as files and directories (folders) • Manages use of storage address spaces • Metadata for identifying data • file name • owner • dates
Connecting, storing and filing as a complete storage system Connecting
NAS and SAN analysis NAS is filing over a network SAN is storing over a network NAS and SAN are independent technologies They can be implemented independently They can co-exist in the same environment They can both operate and provide services to the same users/applications
Protocol analysis for NASand SAN Filing NAS SAN Network Storing Connecting
Integrated SAN/NAS environment NAS Server+SAN Initiator “NAS Head” Filing Storing Connecting Connecting
Common wiring with NASand SAN NAS Head Filing Storing Connecting
Lesson #2 The I/O path
Host hardware path components MemoryBus System I/O Bus Storage Adapter (HBA) Memory Processor
Host software path components Application Multi-Pathing VolumeManager Device Driver Filing System CacheManager Operating System
Network hardware path components Switches, hubs, routers, bridges, gatways Port buffers, processors Backplane, bus, crossbar, mesh, memory Cabling Fiber optic Copper
Network software path components Access and Security Virtual Networking FlowControl Fabric Services Routing
Subsystem path components Access and Security NetworkPorts Cache Resource Manager Internal Busor Network
Device and media path components Disk drives Tape drives Tape Media Solid state devices
The end to end I/O path picture Storage Adapter (HBA) Device Driver CacheManager System I/O Bus MemoryBus VolumeManager Memory Filing System Multi-Pathing Operating System App Processor Virtual Networking FlowControl Access and Security Fabric Services Routing Network Systems Cabling SubsystemNetwork Poirt Access and Security Disk drives Tape drives Resource Manager Internal Busor Network Cache
Storage subsystems Lesson #3
Generic storage subsystem model Controller (logic+processors)Access control Resource manager Storage Resources NetworkPorts Internal Bus or Network Cache Memory Power
Redundancy for high availability • Multiple hot swappable power supplies • Hot swappable cooling fans • Data redundancy via RAID • Multi-path support • Network ports to storage resources
Exportedstorage Exportedstorage Physicalstorage device Physicalstorage device Exportedstorage Exportedstorage Physicalstorage device Physicalstorage device Exportedstorage Exportedstorage Exportedstorage Exportedstorage Physical and virtual storage Subsystem Controller Resource Manager (RAID, mirroring, etc.) HotSpare Device
SCSI communications architectures determine SAN operations • SCSI communications are independent of connectivity • SCSI initiators (HBAs) generate I/O activity • They communicate with targets • Targets have communications addresses • Targets can have many storage resources • Each resource is a single SCSI logical unit (LU) with a universal unique ID (UUID) - sometimes referred to as a serial number • An LU can be represented by multiple logical unit numbers (LUNs) • Provisioning associates LUNs with LUs & subsystem ports • A storage resource is not a LUN, it’s an LU
SCSI LU UUID B SCSI LU UUID A SCSI LU UUID C SCSI LU UUID D Physicalstorage devices Physicalstorage devices Physicalstorage devices Provisioning storage LUN 0 LUN 1 Port S1 LUN 1 LUN 2 Port S2 LUN 2 Port S3 LUN 3 LUN 3 Port S4 LUN 0 Controller functions
SCSI LU UUID A Path 1 Path 2 Multipathing LUN X MP SW LUN X
ExportedVolume ExportedVolume Controller Cache Manager ExportedVolume ExportedVolume Caching Write Caches 1. Write Through (to disk) 2. Write Back (from cache) Read Caches 1. Recently Used 2. Read Ahead
Tape subsystems TapeDrive TapeDrive Tape Subsystem Controller TapeDrive TapeDrive Robot Tape Slots
Subsystem management Now with SMIS Management station browser-based network mgmt software Ethernet/TCP/IP Out-of-band management port In-band management ExportedStorage Resource Storage Subsystem
Data redundancy 2n n+1 Duplication Parity Difference -1 d(x) = f(x) – f(x-1) f(x-1)
Duplication redundancy with mirroring Host-based Within a subsystem I/O PathA MirroringOperator I/O Path I/O PathB Terminate I/O & regenerate new I/Os Error recovery/notification
Duplication redundancy with remote copy Host Uni-directional (writes only) A A B A A
Point-in-time snapshot Subsystem Snapshot Host A A B A A C
Lesson #4 RAID, volume managementand virtualization
RAID = parity redundancy 2n Duplication Parity Difference n+1 -1 d(x) = f(x) – f(x-1) f(x-1)
History of RAID • Late 1980s R&D project at UC Berkeley • David Patterson • Garth Gibson (independent) • Redundant array of inexpensive disks • Striping without redundancy was not defined (RAID 0) • Original goals were to reduce the cost and increase the capacity of large disk storage
Benefits of RAID • Capacity scaling • Combine multiple address spaces as a single virtual address • Performance through parallelism • Spread I/Os over multiple disk spindles • Reliability/availability with redundancy • Disk mirroring (striping to 2 disks) • Parity RAID (striping to more than 2 disks)
ExportedRAIDdisk volume (1 address) Capacity scaling Storage extent 1 Storage extent 2 Storage extent 3 Storage extent 4 Combined extents 1 - 12 Storage extent 5 Storage extent 6 Storage extent 7 Storage extent 8 Storage extent 9 Storage extent10 Storage extent11 Storage extent12 RAID Controller(resourcemanager)
1 6 5 2 3 4 Performance RAID controller (microsecond performance) Diskdrive Diskdrive Diskdrive Diskdrive Diskdrive Diskdrive Disk drives (Millisecond performance)from rotational latency and seek time
Parity redundancy • RAID arrays use XOR for calculating parityOperand 1 Operand 2 XOR Result False False False False True True True False True True True False • XOR is the inverse of itself • Apply XOR in the table above from right to left • Apply XOR to any two columns to get the third
Reduced mode operations • When a member is missing, data that is accessed must be reconstructed with xor • An array that is reconstructing data is said to be operating in reduced mode • System performance during reduced mode operations can be significantly reduced XOR {M1&M2&M3&P}
Parity rebuild RAID Parity Rebuild • The process of recreating data on a replacement member is called a parity rebuild • Parity rebuilds are often scheduled for non-production hours because performance disruptions can be so severe XOR {M1&M2&M3&P}
RAID 0+1, 10 Hybrid RAID: 0+1 RAID Controller Diskdrive Diskdrive Diskdrive Diskdrive Diskdrive Diskdrive Diskdrive Diskdrive Diskdrive Diskdrive 1 2 3 5 4 Mirrored pairs of striped members
Volume management and virtualization • Storing level functions • Provide RAID-like functionality in host systems and SAN network systems • Aggregation of storage resources for: • scalability • availability • cost / efficiency • manageability
OS kernel File system Volume management • RAID & partition management • Device driver layer between the kernel and storage I/O drivers Volume Manager Volume Manager HBA drivers HBAs
Volume manager Server system Volume managers can use all available connections and resources and can span multiple SANs as well as SCSI and SAN resources Virtual Storage SCSI disk resource SCSI HBA SCSI Bus SAN disk resources HBA drivers SAN cable SAN Switch SAN HBA
SAN storage virtualization • RAID and partition management in SAN systems • Two architectures: • In-band virtualization (synchronous) • Out-of-band virtualization (asynchronous)