SAN Management and Application

1. 21세기 유망핵심부품 기술 세미나 SAN Management andApplication 2001. 9. 11 성균관대학교 저장시스템 연구실 윤 희용

2. Outline • SAN Architecture and Model • SAN Technology Trends • SAN Management Architecture • SAN Applications • SAN File System • SAN Standardization • SAN Hot Issues • Conclusion

3. SAN Architecture • Extract the value of data • Backup / Recovery • Outboard data movement • Clustering, data sharing SAN Application SAN Management • Zero latency, one-touch management • Centralized local point • Reduced cost • Flexibility and automation Services and Support Solutions SAN Interface • Migration paths • Investment protection • Fibre channel • Distance, speed, reliability • Hubs, switches • Any storage to any system SAN Hardware Base Storage • SAN from initiative to implementation • Reduced time & cost to implement • Skilled resources • SAN integration, testing, & verification • SAN-enabled • Native attachments • SAN-aware

4. Database(dbms) File system(fs) Host Network Block aggregation Device Storage Model Application File/record layer Services Block layer Storage domain Discovery, monitoring Redundancy mgmt (backup, …) High availability (fail-over, …) Resource mgmt, configuration Security, billing Capacity planning Storage devices (disks, …)

5. 1. Direct-attach 2. SN-attach 3. NAS server Host Host 4. NAS head File metadataserver LAN Host. with LVMand software RAID Host. with LVM 5. SN aggregation NASserver 6. Metadata aggr’n NAS head Host block-aggregation Aggregationappliance SN SN block-aggregation Blockmetadataserver Disk array Device block-aggregation Storage Model – Full function View Application File/recordlayer Blocklayer

6. Host-to-host SN appliances Device-to-device Host block-aggregation Aggregationappliance Aggregationappliance SN WAN WAN WAN SN block-aggregation Disk array Disk array Device block-aggregation Storage Model – Multi-site Block Storage Functions: point-in-timecopy, caching, local & remote mirroring, … Application Application Host. with LVM Host. with LVM

7. Doubling every 8-9 months opticalnetworksspeed Doubling every 12 months Doubling every 18 months storagecapacity computingpower Why Networked Storage? Moore’s Law

8. IP Networking External Storage Optical Networking Storage and Network Convergence Storage Networking

9. Technology Trend – Fibre Channel • Fibre Channel provides new levels of Bandwidth, Connectivity, and Flexibility • Switches provide non-blocking 100 MB/sec (1 Gbit/sec) per port with any-to-any connectivity • Long-wave ports provide up to 10 km distance • Future Developments: • Large port-count switches (~ 256) • Fibre Channel has grown to 2 Gbit/sec • 10 Gbit/sec in 2-3 years

10. REMOTE FC-SW CAMPUS FC-SW Global IP-SAN 1,000+ Km FC-SW SWITCHMAN 100Km FC-AL SWITCH HUB FC / ISCSI DWDM 1-11 KM FC-AL (Point-to-Point) 500M FC-SW FCAL SWITCH SCSI25M SWITCH ESCON / BUS & TAG FC -SW 1990 1994 1997 1998 1999 2001 2002+ SAN Connectivity Evolution DATA CENTER FC -SW

11. Clients LAN Client Network MVS UNIX NT Heterogeneous Servers Conventional IT Organization Network used for exchange of data between clients and servers and between servers themselves. Difficult to manage, wasted resources, often unprotected: Impossible to scale

12. Clients LAN Client Network MVS UNIX NT Heterogeneous Servers IT Organization with Enterprise Storage Network used for exchange of data between clients and servers and between servers themselves. Enterprise Storage provides storage and exchange of data by servers Enterprise Storage Simpler to manage, resources easily re-deployed, superior protection. The next stage of manageability and scalability

13. MVS IT Organization SAN Data Presentation Information visualization using HTML, Java, etc. Clients LAN Client Network NAS Appliance UNIX NT Heterogeneous Servers Data Logistics Information MoveBackup/RestoreInformation Mgmt Cluster Support Storage Area Network

14. Why Networked Storage? • Consolidation - Application Server and Storage • Storage Consolidation • Management - Total Cost of Ownership (TCO) • Disaster recovery - Backup / Restore • Availability • 7 x 24 xForever • Flexibility • Rapid adaptation of changes in business requirements • Capacity Expansion • Performance • Configuration

15. Shift in Customer Perceptions Historical View New Paradigm 1) IT saves dollars IT contributes to revenuegrowth 2) Choose technology first Put information needs first 3) Managing technology Managing information 4) Acquisition costs Total cost of ownership 5) Rigid infrastructure Change-tolerant infrastructure Technology / Vendor lock-in Flexibility, compatibility

16. SNIA Management Standards - 1999 • Common Information Model (CIM) • Simple Network Management Protocol- Management Information Bases (SNMP-MIB) • Reviewed and assisted in the standardization of two SNMP MIBS • Fibre Alliance MIB in process within IETF • Creation of new MIB for Storage Libraries(First MIB to use CIM schema) • SNIA CIM schema definitions submitted for CIM V2.2 • Storage Media Library (SML WG) • Disk Resource Management (DRM WG) • Fibre Channel (FC WG) • October 1999 Storage Networking World • First demonstration of V2.2 CIM web based technology

17. SNIA Management Standards – 1st half 2000 • Education • Release of the first Storage Networking Dictionary to the industry • Successful completion of three major industry events: • The SNIA workgroup conference - Feb 2000 • Spring Storage Networking World - April 2000 • CIFS 2000 Conference and Interoperability Lab - May 2000 • Management • Delivery of Industry first Open Source - CIM-OM(CIM-Object Manager) • Disaster Recovery - Backup • SCSI-3 extended copy command • Submitted to ANSI T10 • Driving the NDMP version 4 development

18. SNIA Management Standards – 2nd half 2000 • Education • The SNIA workgroup Symposium - Sept. 2000 • Vendor and IT Consumer Surveys • Fall Storage Networking World 2000 • Architectural white papers • Interoperability • Continued Standardization of CIFS (Common Internet File System) • Test plans and reference implementations for new and existing standards • Management • Adoption of the SNIA developed HBA-API by T11 • Management models for extended copy sessions • MIBs and extensions to the CIM Schema forstoragemanagement • Disaster Recovery • APIs for snapshot and checkpoint

19. SNIA Management Standards – 2001 • Grand Opening of the SNIA Technology Center Jan ‘01 • SNIA-Japan - SNW Japan Jan. ‘01 • SNIA Shared Storage Model March ‘01 • Spring Storage Networking World April ‘01 • Spring SNIA Work Group Symposium April ‘01 • Summer SNIA Work Group Symposium June ‘01 • SNIA CIFS Conference Aug ‘01 • European Storage Networking World Sept. ‘01 • Fall Storage Networking World USA Oct. ‘01

20. SAN Management Architecture

21. SAN Management Architecture – EMS & CMS • EMS • SNMP based service • DMI based service • FC service • SES service • Native CIM service • CMS • Entity mapping • Common data model • Access services

22. SAN Management Open Standard Platform • SCSI Instruction Set • SCSI Enclosure Services(SES) • SCSI Self-monitoring Analysis and Reporting Technology • (SMART) • SCSI Accessed Fault-Tolerant Enclosure(SAF-TE) • SNMP • Web-based Enterprise Management(WBEM)

23. Storage Management Problem SAN Solution Time for data backup Improvement of backup speed through bandwidth and protocol efficiency Impossibility of remote backup, mirror, or recovery Support remote operation through a cable of about 10km Insufficient solution for local backup and mirroring Ideal platform for hierarchical storage management(HSM) Exhaustion of network bandwidth by LAN connection for server backup Decrease LAN traffic by separating server-storage from LAN connection SAN Management

24. Clustering Data Protection Network Architecture SAN Data Vaulting Replicate Recovery Data Exchange SAN Application

25. SAN Application - Remote Backup

26. SAN Application - Server-Storage Clustering

27. Client Client Client LAN SAN Net server Net server Backup server Disk storage Tape SAN Serverless Backup

28. Client Client LAN SAN Net server Net server Net server Disk storage Disk storage Disk storage Tape SAN Storage Integration

29. SAN Storage Expansion FC-AL

30. server FC host 어댑터 SAN Storage Expansion Fabric

31. Serially connected Individual loop Loop #1 Loop #2 Loop #3 Loop #4 Loop #5 SAN Storage Expansion Multiple Loops

32. NT file server FC-AL w/SCSI FC-AL w/IP Shared storage workstation SAN FC Workgroup

33. server FC switch/hub JBOD or RAID SAN Recovery – Server Clustering

34. SAN Recovery – HA clustering

35. SAN Recovery – Data Mirroring

36. SAN File System • SAN Support • Customizability • According to specific I/O requirement such as web, multimedia • User-Controllability • Users decide logical storage space, data format, fault tolerance, and recovery • High Performance and Scalability • Dynamic load balancing across all disks • Hot system mode change • Storage environment optimization according to I/O requirement • High Availability • Software RAID, Chained-Declustering, Fast Recovery • Compatibility withexisting file system in API and access method

37. SAN File System Requirement • Mass Storageability • Unlimited • High Scalability • Capacity, performance, connectivity, bandwidth, cost • High Availability • Duplex • Online data backup • Easy Manageability • Effective central management and remote control through Internet • High Utilizability • Logical disk pool • Load Balanceability • Data migration through SAN

38. SAN FS MPI-IO Ext FS NTFS HFS SAN File System Structure Application Software (Oracle, Word, Web Browser, etc.) System Call Interface Timer Function Process Management IPC New I/F Ext2 I/F Virtual File System Memory Management Networking Buffer Cache Device Driver New Driver Kernel Hardware (Disk, Memory, CPU, Network Cards, etc.)

39. SAN File System Products SoftWorks Item Veritas IBM Tivoli Target Unix/NT/VMS O S Unix Unix/NT Multiplatform Native File System Native NTFS/UFS New Low Tuning Low No High Possible Parallel I/O Possible No Possible Yes SAN Support Yes Yes Yes Yes Disk share Yes Yes Yes No MPI-IO No No Yes No Linux version No No Yes Yes High Availability Yes Yes Yes SAN Support Key Functionality SAN Support FS Share SAN Support FS Share SAN Support I/O Perf

40. Standard Collaboration • DMTF– WBEM (Web-Based Enterprise Mgmt), CIM • ANSI– T10 (SCSI) and T11 (Fibre Channel) • IEEE– Media management and tape • IETF– SNMP, IP SANs, IP storage,directories/LDAP and policy • And others • FCIA– Fibre Channel Industry Association • Fibre Alliance- SNMP MIB

41. ULP(Upper Level Protocol) SCSI-3 ATM IP IPI-3 Inst mapping (IPI-3 STD) SCSI-3 Inst mapping FC link Encapsulation FC-LE FC-ATM FC-4 FC-3 Common service FC Physical/signaling Interface (FC-PH,FC-PH2, FC-PH3) FC-AL FC-AL-2 FC-2 Frame protocol FC-1 En/decoding 8B/10B Coding Copper/Fibre Physical FC-0 SAN Standardization - FC • 1994 ANSI X3T11

42. SAN – Hot Issues • Storage over Wide Area Networks • SANs for Everyone • Storage Services Providers • Storage Management Applications • Extended Copy for server-free data movement • Host Adapter API for fibre channel management • IP Storage technologies • Fibre channel switch interoperability • Infiniband

43. SAN over WAN

44. SAN for Everyone

45. Storage Services Providers

46. Storage Management

47. IP Storage

48. Switch Interoperability

49. InfiniBand

50. PCI vs IB