1 / 33

Scality RING Organic Storage 4 Jan. 2013

Scality RING Organic Storage 4 Jan. 2013. Agenda. Corporate Profile Industry Trends and Business Challenges Scality RING 4 Scality Solutions Ecosystem Roadmap and Vision Conclusion. Scality – Quick Facts. Founded 2009 Experienced management team HQ in the San Francisco, Global reach

everly
Télécharger la présentation

Scality RING Organic Storage 4 Jan. 2013

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Scality RING Organic Storage 4Jan. 2013

  2. Agenda • Corporate Profile • Industry Trends and Business Challenges • Scality RING 4 • Scality Solutions • Ecosystem • Roadmap and Vision • Conclusion

  3. Scality – Quick Facts • Founded 2009 • Experienced management team • HQ in the San Francisco, Global reach • <50 Employees, 20 engineers in Paris • 24 x 7 support team • US patents • $13M invested in Scality to-date • 120% annual growth • Industry Associations “Aggressive use of a scale-out architecture like that enabled by Scality's RING architecture will become more prevalent, as IT organizations develop best practices that boost storage asset use, reduce operational overhead, and meethigh data availability expectations.”

  4. A Rolling and Unstoppable Ball…

  5. Industry Trends • Rapid unstructured data growth • “Enterprises will see a staggering 52% growth in data over the next years — much of it an increase in unstructured data”, IDC • x8-10 in 5 years, x100 in 10 years !! • 100TB today will be 1PB then 10PB • Rapid decline in storage costs • “Cost of disk storage systems is declining >30% annually”, IDC • Skyrocketing storage budget & complexity • More storage devices need to be managed • Increasing cost of commercial real estate • Increasing energy costs (power, cooling) • Cost of managing storage between 5 and 10 times the acq. cost (Gartner, IDC) StorageBudget TB Growth $/TB Cost

  6. Data and Storage Challenges 100s Millions of Users, 10s-100s PB of Dataand Billions of files to store and serve What all these companies have in common ? Internet/Cloud business impossible to sustain and developwith traditional IT approaches

  7. Scality’s Mission Control and EfficiencyatScale UnifiedScale-Out Storage Softwarefromprimary to long termstoragewithCloud’sadvantages,investment protection and ready for future Their DC YOUR App. YOUR Data YOUR DC YOUR App. YOUR Data Their DC Their App. YOUR Data

  8. Scality RING 4 Email File Storage StaaS Digital Media Big Data* Enterprise &Cloud EmailSystem DataProcessing with Hadoop Scale Out File System Origin Server S3 & CDMI API • Scality RING Organic Storage 4 P2P MD DATA MESANewSQL DB S3CDMI ARC RingTopology End-to-EndParallelism ObjectStorage Replication ErasureCoding GeoRedundancy Tiering Management Standard x86 * Available Q2/2013

  9. Hardware Agnostic x86 server (1U, 2U…4U) with CPU, RAM,Ethernet and DAS running Linux Racks (10…40U) Clusters Scale-Out Storage Software based on Shared Nothing Model

  10. Distributed Architecture • From Servers to Storage Nodes • RING Topology, P2P Architecture • Limitless Scale-Out Storage based on Shared Nothing model • Fully Distributed Storage (Data and Meta-data) 1 node manages 1/36th of thekeyspace Scality RING servers (6) storagenodes (ex: 6/server, total=36) 36 storage nodesprojected on a ring P2P

  11. Distributed Architecture • DHT & Consistent Hashing • 360° key space, each node manages a key range • Key generation and projection (no overhead at running time) • MIT CHORD Inter-routing Algo. + 3 Scality methods (Proxy, Balance & Rebuild) • 1/6 key space to rebalance when a storage node is lost • No Single Point of Failure, No central Catalog or DB • Stateless architecture • Easy to grow, high resistance and seamless to failures • Self Healing and Organic Upgrade • Elastic Clustering • Inherent Load balancing & Symmetry • Every node can serve a request P2P

  12. End-to-End Parallelism APPLICATIONS / CONNECTORS APPLICATIONS / CONNECTORS APPLICATIONS / CONNECTORS APPLICATIONS / CONNECTORS APPLICATIONS / CONNECTORS • Parallel Connectors accessto Storage Nodes • Performance aggregation • Redundant Data Path • Multiple Storage Nodesper server • Minimum 6 to increase parallelism and data independence • Fast and easy rebuild • Multiple IO Daemonsper Server • Control Physical and Boost IO STORAGE NODES STORAGE NODES STORAGE NODES STORAGE NODES I/O DAEMONS I/O DAEMONS I/O DAEMONS I/O DAEMONS I/O DAEMONS I/O DAEMONS SSD TIEREDSTORAGE SATA Scality Parallelism Factor #Storage Nodes x #IO DaemonsvsSimple server node with only 1 IO Engine Independent Performance and Capacity Scalability

  13. Object and Key/Values Stores 160 bits • Object as a opaque entity with Metadata and Data • No limit for object size and number • Random block R/W access on large objects • Object Chunking, Splitting, Striping and Streaming • 160 bit Key Space with 128 bit Object key • Store node acting as Simple and High PerformanceKey/Value stores • Fully Autonomous 8 bits Class 24 bits Dispersion Payload Class of Service 128 bits Replica DATA MD

  14. Internal Distributed DB Main • Scality MESA • NewSQL Philosophy • Distributed Tables across Storage Nodes • Multiple Indexes per table • Schemas updates/Flexible schemas • 100% ACID • 100% Elastic • Linear Transactional Performance • Automatic Fault Tolerance • SQL front-end (not exposed) DEFS Mapping Table Page -> Properties/Name Key Page Key Key Key Page # Page # Page # Page Page Key Key Key Key Key Key Value Value Value Value Value Value

  15. Data Replication • No data transformation • Clear/Native data format • Very fast access • Simple projection • Class of Storage • Up to 5 replicas (6 copies) • Rack-aware • Guarantee of full independent object location • Self-healing • Balance misplaced objects • Transparently proxy misplaced objects • Rebuild missing replicas • Permanent CRC of all contents (no silent data corruption) 11 00 220 20 200 40 180 60 171 80 160 91 140 100 120

  16. ARC: Scality’s Erasure Coding • RAID 5 or 6 can’t scale without risk • With Bit Error Rate vendors information (1 bit error among 1014 bits) • 55% of probability to receive an error for a volume of 10TB • 99.9% for 100TB !! (worst beyond) • Replication is a great method but too costly for large storage environment • Solution: Scality Advanced Resilience Configuration (ARC) • Available as a standard RING Core feature (no extra cost) • Configuration option, running within the RING • Leveraging Reed Solomon technology ARC

  17. Scality ARC • Fully configurable, by default ARC(14,4) meaning… • 14 data fragments + 4 checksums fragments so max. loss of 4 elements in a transaction (18 total fragments committed on disk) • Read() is satisfied with minimum of 14 fragments: (14,0), (13,1), …, (10,4) • Data fragments = Native Data (no transformation) • Direct and fast read access ScalityRING 4 Checksums Scality ARC(14,4) ARC • HW overhead • ratio = 1.3 • (14+4/14) Data inputs … Data 14 Data source Output to be stored • Better Durability & Reduced Storage overhead than Replication

  18. Scality ARC Comparison A A B A-B B A A+B B A B A+B A*B Replication Scality ARC Dispersed ARC • Reduced storage space • Redundant information • Direct access Scenario #1: loss of A read(A+B) - read(B) = A Scenario #2: loss of B read(A+B) - read(A) = B • Reduced storage space • Scrambled data • Latency with additionalcomputation for each read(decoding phase) • Storage space x3 • Multiple copies • Direct access

  19. Geo Redundancy • Business Continuity with “true 99.999%” availability including maintenance • Multi-site topology with Scality RING (up to 6 sites) • Replication or Geo Erasure Coding implementation – Synchronous • Or Multi-RINGs on Multi-site (independent topology) – Asynchronous Asynchronous multiple independent RINGs Synchronous stretched RING across 2 sites

  20. Auto Tiering 20% Tier 1 Primary Site Site A • Tiering like HSM • Fully automated and transparentfor application • 20/80 approach • Perfect for Hot data on SSD/Flashand Cold data on SATA • Criteria • Age, Access Time & Object size • Potentially different data protectionmechanisms • Replication with 3 copies on Tier 1 • Erasure Code – Scality ARC – on Tier 2 Data Migration across RINGs within same site or across sites 80% Tier 2 Secondary Site Site B

  21. Access Methods and Standards • Open Cloud Access Strategy • Scality RING as the storage backend • File and Object Access Methods • Local and Remote • Global Namespace • Scality RS2 API (HTTP/REST) • Compatible with Amazon S3 • Full CDMI server implementation • By object ID and hierarchical path-based namespace • Cloud Files as the OpenStack Swift protocol S3 CDMI Internet NFS NFS NFS SOFS SOFS S3CDMI Scality RING

  22. Management • Supervisor • Central management platform • Monitor application connectors & storage nodes down to individual disk drives • Passive component • Detailed Activity Log • RingSH • Command line interface • Easy to integrate or script with • Manage platform, store/retrieve/delete objects • List all keys

  23. « ExceptionalPerformance » “ESG Lab verified exceptional performanceforan object-based storage solution, which rivalsblock-based solutions. Aggregate throughput scaled linearly as nodes were added to a RING. Response times improved as the RING grew in size, allowing for predictability when deploying a RING.”

  24. Scality RING – The BIG picture

  25. Scality Solutions • Scality RING CoreP2P, Replication, Erasure Code, Tiering, API/Interface (HTTP/REST with Scality SRWS and RS2 Light, CDMI by Obj. ID), Management • File Storage • Scale-Out File System for Linux • NFS/CIFS • CDMI by Path • Global Namespace • GeoSync • StaaS • Amazon S3 API • Authentication, Metering • Option Multi-Geo • Sync-n-Share1 • Digital Media • Optimized SOFS for CDN Origin server • Big Data2 • Hadoop integration with in-place data processing • Scality SOFS instead of HDFS • Email • Full Email platform integration • Certified with Openwave Messaging, Critical Path, Cyrus, Dovecot, VMware Zimbra, Open-Xchange Open Cloud Access S3 & CDMI API 1: Product available separately / 2: Available Q2/2013

  26. Scality Ecosystem and Partnership File Sync & Share Backup& Archive Gateway Email Hardware vendor strategic partnerships

  27. Scality Product availability • Software only • Hardware agnostic • RING Core + Solutions Pack • Multi-Geo option • Price per usable storage capacity • “Appliance” based on MIS(Duo or Solo server) • RING Core + StaaSPack • Multi-Geo option • Price per enclosure (4U) • Highest storage density on the market (72 x 4TB/4U) Scality RING Organic Storage4.0

  28. Release Calendar & Roadmap • H1/2014 TBC • RING 5.0 • “Universal • Data Platform” • - Block interface • - Policy Engine & Data Placement • June 2012 • RING 4.0 • - ARC (Erasure Code) • SOFS 1.0 (FUSE) • Supervisor (Keys, Agent, SNMP) | | 2012 2013 2014 • Q4 2013 • RING 4.3 • OCA Phase 3 (Metering/Multi-Tenant + Swift + S3) • Act./Act. Geo RS2 • Q2 2013 • RING 4.2 • SOFS 2.0 (NFS, CIFS) + GeoSync • Hadoop (HDFS replacement, computing on nodes) • Multipart Upload • Q1 2013 • RING 4.1 • Digital Media • Open Cloud Access (CDMI & SOFS) • SOFS GeoSync • Supervisor (usage) • OpenStack integration(EBS with Cinder) Feb. 2013

  29. Scality Technology Milestones Blob Store wth Replication and Tieringfor primary storage (HTTP, S3) Authentication, Metering,Security for S3 Policydata placement, data protection Hadoop for Big Data CIFSbased on Samba 4 Erasure Codedata protection Cloud FilesSwift Protocol | | | | 2010 2011 2012 Q2/2013 Q4/2013 NFS CDMI File Sync-and-ShareOEM Nomadesk Distributed Database Scale Out File System for Linux SQL front-end for Distributed DB Block Storage (to add storage to VMs)with OpenStack integration via Cinder Multi-Tenancy, QoSMgmt(IOPS, Throughput, Capacity) & Snapshot Mature Ready Beta Alpha Prototype Design

  30. Scality Technical Vision Compute Media HPC StaaS Note: Underlined items are roadmap Unified Storage Interface (File, Block, Object and HDFSLocal and Remote) Tenant A Tenant B Tenant C Tenant D Metering Scale Out Access Layer Auth. Real-time Policy & QoS Big Data Analytics Content Indexing Security Storage Back-end Local, Distant or Multi-Geo RINGs Statistics High Perf SATA SSD Erasure Code SATA Long-TermStorage Standard Storage Discovery Customer IT Infrastructure Transactional 100 years 1 year

  31. Conclusion • Technology Leader • Software only solutionwith oem agreements • Proven deployed solution • 100% Data Availability and Durability • Fully Autonomous • High Performance and strong Security • Cost efficiency and rapid ROI UnifiedScale-Out Storage Software

More Related