ElasTraS: Database for the Cloud

1. ElasTraS: Database for the Cloud CSE880 Professor Pramanik Shayne Rashedan & Chongguang Bi 1

2. Agenda • Background • Motivation and Challenges • Examples • Architecture • Schema Level Partitioning • Live Database Migration • Evaluation 2

3. Pay per use model 3

4. What is cloud computing Software as Service, Apps are located in Cloud, software experiences are delivered through the internet Platform as Service, Internet based platforms for developers who want to create services but don’t want to build their own cloud Infrastructure as Service, Cloud users rent storage, computation and maintenance from cloud providers (pay per use utilities) Data as Service, Infrastructurefor web-scale data mining and knowledge discovery 4

5. Large numbers of application databases (or tenants) with diverse schemas, unpredictable load and transactions. Scaling out using clusters of commodity servers and sharing resources among tenants (multitenancy) Pay-Per-Use: paying a small subscription fee on per usage basis every time the software is being used (e.g. hourly) Elastic Scaling: When the load increases, add more servers to the database tier to distribute the load, and vice versa. Fault-tolerant, self-managing and highly available to support mission critical applications Challenges 5

6. ElasTranS: Elastic, Scalable, Self-Managing 6

7. Key-value stores Bigtable, PNUTS and Dynamo, serving hundreds of thousands of concurrent requests Schema Level Partitioning Microsoft SQL Azure ,Relational Cloud and a prototype from Yahoo!, standard RDBMSs (such as SQL Server or MySQL) Decoupled shared storage Project Deuteronomy and Hyder, persistent data layer is separate from the logic that executes operations on that data,on the other hand in Amazon EC2, each node is data node and the transaction processor Examples: 7

8. Live data migration VM pages, iterative phase-by-phase copying pages, migrates live processes and network connections using OS and networking primitives Zephyr, for shared nothing database architectures such as Cloud SQL Server or Relational Cloud. The persistent data is stored in disks locally attached to every node. Hence, the persistent data must also be migrated Advanced transactions at Scale, Megastore, Percolator, ecStore and G-Store Sinfonia, two phase commit (2PC) protocol to guarantee atomicity of operations in a distributed set of nodes Examples: 8

9. Transaction Manager Row Groups Architecture of ElasTraS Server Owning Transaction Manager Partitions of Data 9 Append-only Multiversion Data

10. Routing OTM Routing Architecture of ElasTraS Routing Data Storage Data Storage OTM OTM Routing Routing Data Storage OTM Data Storage User / Client TM Master Routing 10

11. Static Partitioning 11

12. Online Multi-player Game 12

13. Row Groups 13

14. Row Groups 14

15. Row Groups = Partitions 15

16. Why TPC-C Tree Schema? When amounts of data and transactions are large, assign some jobs to more nodes…. Assign all data and transactions related to Michigan State University Football Team to another node? How?? 16

17. Why TPC-C Tree Schema? Partition Keys 17

18. Distribute Transactions? Snapshot & Multiversion Data Problem 1 Read A Read Lock? Write Lock? Manager Data Read B Problem 2 Read Manager Data A Write B 18

19. Snapshot & Multiversion Data Problem 1 Just Read Snapshot! Read A OTM Data Read B Problem 2 Just Read Snapshot! Read OTM Data A While “A”s are reading, B can update database. Write B 19

20. Albatross Migration Timeline Source Ownership Destination Ownership Steady State 1.Begin Migration 2.Iterative Copying 3.Atomic Handover Steady State Clients TM Master 20

21. Minimum node unavailability achieved by iteratively copying database cache and state of active transactions Elastic Load Balancing, rule-based model, if OTMs load is above the threshold, new OTMs are added, if load is below a min-threshold, partitions are consolidated into a smaller number of nodes. Partition assignment algorithm optimization problem with Greedy Heuristic based on load statistics of partitions Atomic ownership handover, ensure efficient and ACID execution of transactions, transactions must execute during migration, synchronizing commit logs of two nodes Aborted transactions, minimize loss of data, restart aborted transactions in destination Failure Handling, how to recover migration state after failure Challenges 21

22. User Data Elastic Load Balancing User Data User OTM OTM OTM OTM TM Master Data Data 22

23. Benchmarks: Yahoo! Cloud Serving Benchmark Key-value stores 3 tables, 10 VARCHAR(100) column per table TPC-C Benchmark 9 tables describing wholesale supplier 5 Transactions: NewOrder, Payment, OrderStatus, Delivery, StockLevel Evaluation 23

24. Comparing with Albatross: S&M: Stop and Migrate stop serving, flush, transfer ownership F&M: Flush and Migrate flush, stop serving, transfer ownership Evaluation 24

25. User Data User Migration Elastic Load Balancing User User Data Data Data OTM OTM TM Master OTM OTM Data Data 25

26. Evaluation Albatross: Low Unavailability Low Latency 26

27. Evaluation Albatross: Almost no additional cost for read operations Migration costs much time 27

28. Evaluation Albatross: Low Latency Long Migration Time 28

29. Not Enough Scale Evaluation Scale is Large Enough Load is Low Migrate to Low Scale 29