Designing scalable applications for cloud

1. Designing scalable applications for cloud Raghavan Subramanian, Associate Vice President, Head of Cloud-computing CoE, Infosys

2. Agenda • Overview of scalability • Scale-up and Scale-out • Cloud design considerations • Key principles of scale-out design • A few techniques for transforming a scale-up design into a scale-out design • Scale out Apps on Windows Azure

3. Overview of Scalability • Scalability • Is a property of a system, a network, or a process, which indicates its ability to either handle growing amounts of work in a graceful manner or its ability to be enlarged. • For example, it can refer to the capability of a system to increase total throughput under an increased load when resources (typically hardware) are added.

4. Scale up/Vertical scalability • Scale up/Vertical scalability is adding more memory and CPUs to a single box. • One can realize its benefits in scenarios like • Large shared memory space, • Many dependent threads, • Tightly-coupled internal interconnect • Small scale apps

5. Scale Out/Horizontal scalability • Scale Out/Horizontal scalability is adding more boxes of similar memory and CPU • One can realize its benefits in scenarios like • Work which can be broken up into smaller tasks • Activity which can run independently and as an Atomic unit • Process large volumes of data

6. Law of diminishing returns • In economics, diminishing returns (also called diminishing marginal returns) refers to how the marginal production of a factor of production starts to progressively decrease as the factor is increased. • According to this relationship, in a production system with fixed and variable inputs (say factory size and labor), each additional unit of the variable input (i.e., man-hours) yields smaller and smaller increases in outputs, also reducing each worker's mean productivity. • Consequently, producing one more unit of output will cost increasingly more (owing to the major amount of variable inputs being used, to little effect). • This concept is also known as the law of diminishing marginal returns or the law of increasing relative cost.

7. Concept of Linear Scalability • Linear scalability, relative to load or demand, means that with fixed resources, performance decreases at a constant rate relative to load or demand increases. • Linear scalability, relative to server resources, means that with a constant load or demand, performance improves at a constant rate relative to changes in resources. Roughly Linear Scalei.e. Additional throughput achieved by each additional unit remains constant Throughput Non Linear Scalei.e. Additional throughput achieved by each additional unit decreases as more are added Computation Units

8. Scale up Vs Scale Out: which one scores more

9. Cloud encourages the usage of scale out design by solving some of the scale out issues.. • On-demand provisioning • Infrastructure management outsourced to the cloud vendor • Elastic scale – Go up or down instantaneously • Higher level of abstractions with fault-tolerance and resilience built-in • Pay-per-Use- Optimize capital expenditure (Hardware and Software) However if apps don’t scale within the data center they certainly won’t scale when just moved to the cloud

10. Key Principles of Scale out design • Asynchronous, event-driven design • Parallelization • Divide and Conquer • MapReduce /Master-Worker • Idempotent operations • De-normalized, partitioned data (sharding) • Shared nothing architecture • Go Stateless • Fault-tolerance by redundancy and replication (Design for failure)

11. Typical Scale-out design pattern X Task A Task B Task C Task D Controller Publish 5X Event Driven Parallel Idempotency Task A Task B Task C Task D Status Watch Merge Publish Task A* Task B* Task C* Task D* Status Watch Merge Publish Data Data Dead-letter Q Data Compensate Sharding Fault-Tolerance X 3X

12. Techniques to transform a scale-up design to a scale-out design • Analyze the existing code flow - Prepare diagrams – flowcharts, components diagrams • Identify logical units of work and extract them into separate components which can work in an independent fashion – Think Contract First design • Each thread can be considered as a separate task • Extract common functionality to a shared service • Make component interaction message or document driven • Consolidate Public variable to messages/entities • Implement the DTO patterns • Re-arrange the code flow - Activity and State diagrams • Parallelize • Asynchronize • Choose NoSQL over RDBMS • De-normalize data to reduce firing multiple join-like queries • Partition data to form smaller data sets which can Shard • Implement message queuing to build loosely coupled components • Question yourself, “What if this fails?”. Put your failover mechanisms in place • Dead letter queues • Retry Handlers • Alerts/Notifications

13. Scale-out Apps on Windows Azure LB Jobs Windows Azure • Best Practices • Break jobs into smaller chunks so as to avoid inefficient workload distribution • Spawn multiple threads per role to speed up job execution • Rationalize tasks within fewer roles to have more redundancy at same cost • For handling large datasets, store dataset file in Blobs with reference information contained in messages pushed to Queues • For application operating in phases, store transient messages in Queues and statuses of job/messages in Tables • Use affinity groups – keep compute roles and storage closer • For debugging, instrument code to trace code execution paths across the application LB LB Web Role Web Role Web Role Web Role Web Role Web Role Web Role Web Role Worker Role 3 Web Role Worker Role 1 Worker Role 2 Storage Blobs Tables Queues Windows Azure Components Compute – Web and Worker Role Storage – Blobs, Table and Queues

14. Thank YouAuthored by:1. Bhausaheb Jadhav2. Raghavan Subramanian3. Sidharth Subhash Ghag4. Sonal Arora