Tips & Tricks: Low Latency Web Applications on the Windows Azure Platform

1. COS401 Tips & Tricks: Low Latency Web Applications on the Windows Azure Platform Matthew Kerner Principal Program Manager Lead Microsoft Corporation

2. Low Latency Web Applications • What do I mean? • Global user base, snappy response times • May scale out to handle many concurrent requests • Heavy CPU, memory, disk and network bandwidth requirements • How does this apply to the cloud? • Windows Azure applications are mostly like on-premises applications… • … but some things are different (and even easier) in the cloud. • I will assume you already know the basics of Windows Azure • Agenda • Part 1: Asynchronous design patterns • Part 2: Managing data access • Part 3: Tuning application performance

3. Part 1: Asynchronous Design Patterns

4. Synchronous Design Pattern • Handle one request at a time for each thread • Block on “the work” done for each request, then respond & repeat • Each thread services next request only after first completes • The thread count is the scalability bottleneck • Poor scaling: adding another thread enables only one additional concurrent transaction SQL Azure Web App Front End “The Work” #1 Client Request #1 Middle Tier Client Response #1 WA Storage Thread Thread Response #1 blocks Client Request #2 Waiting… Time passes…

5. Asynchronous Design Pattern • Handle one request at a time for each thread • Queue “the work”, then either handle the next request or return results to the client • Threads can handle additional requests while previous requests are in-flight • The bottleneck is no longer the thread count • Good scaling: adding additional threads enables > 1 concurrent transaction • Caveat: client requests tracked explicitly in app’s data structures • Consider context overhead and contention on thread-safe data structures SQL Azure Web App Front End Client Request #1 “The Work” #1 Middle Tier Client Response #1 WA Storage Response #1 Thread Thread Client Request #2 “The Work” #2 Context Client Response #2 Response #2

6. Asynchronous All the Way • Every tier of your application should be asynchronous • Each tier pushes the bottleneck deeper into the application • SQL Azure and Windows Azure Storage both support asynchronous I/O • ADO.NET Entity Framework • OData (WCF Data Services) • LINQ to SQL • Plain old ADO.NET

7. A Note on Failures • Windows Azure is designed to run on commodity hardware • Eliminate hardware redundancy to reduce costs • Failures can happen at any time • Local in-memory or on-disk data is not durable • You must design redundancy into software • Windows Azure Storage Queues are useful for async communication between role instances • Gracefully handle loss of role instances with message invisibility support • Built-in load balancing • Work items can be executed multiple times: must be idempotent 1 Queue BE FE 3 3 2 2 1 1 BE FE 3 1 2

8. Part 2(a): Managing Client Data Access

9. Get Out of the Way • How to transfer data efficiently to and from clients? • There are different kinds of data; each has its own tricks • Trick #1: Get out of the way when you can • Send clients directly to blob storage for public static content • Media (e.g. images, video) • Binaries (e.g. XAP, MSI, ZIP) • Data files (e.g. XML) Blob Storage Hosted Compute Hosted Compute

10. Shared Access Signatures • What about private content? • Trick #2: Shared access signatures provide direct access to ACLed content • Can be time-bound or revoked on demand • Beware leaking the SAS to adversaries • Also works for write access (e.g. user-generated content) • http://blog.smarx.com/posts/shared-access-signatures-are-easy-these-days 2. Service prepares a Shared Access Signature (SAS) to X using the securely stored storage account key [The caller who presents this URL is allowed to access X between times A and B] Storage Key 1. “I am Bob & I want X” Hosted Compute Stg Key 3. Service returns SAS (signed HTTPS URL) Blob Storage 4. Bob uses SAS to access X directly from Blob Storage for reduced latency & compute load Non-public blob (e.g. paid or ad-funded content) X

11. Serve Blobs from the Edge • Trick #3: Serve public blobs from the edge with the Windows Azure CDN • Use the CDN if you expect multiple reads before content expiration • Reduces cost, latency and load on central storage account Possibly many hops or poor links Few hops CDN Blob Storage Closest Point of Presence Public container X X Blob header determines time-to-live at the edge DNS name resolves to closest POP

12. Windows Azure Content Delivery Network • 24 global locations • Enabling CDN access for your Windows Azure storage account • Enable the CDN in the dev portal • It will generate a new URL for CDN-based access to your account • CDN URL served from edge: http://azXXXX.vo.msecnd.net/images/myimage.png • WA Storage URL served from source: http://myacct.blob.core.windows.net/images/myimage.png • CNAME mappings supported • http://blog.smarx.com/posts/using-the-new-windows-azure-cdn-with-a-custom-domain • Smooth streaming CTP is coming this summer

13. Effective Caching vs. Stale Content • Will the CDN serve stale content? • Default behavior is to fetch once and cache for up to 72 hrs • Modify cache control blob header to control the TTL • x-ms-blob-cache-control: public, max-age=<value in seconds> • Think hours, days or weeks • Higher numbers reduce cost and latency via CDN & downstream caches • Use versioned URLs to expire content on-demand • Enables easy rollback and A/B testing HTML Served by App Blob Storage CDN … <imgsrc="http://azXXXX.vo.msecnd.net/images/logo.2010-08-01.png" />… … <imgsrc="http://azXXXX.vo.msecnd.net/images/logo.2010-10-29.png" />… logo.2010-08-01.png logo.2010-08-01.png logo.2010-10-29.png logo.2010-10-29.png

14. CDN for Web Apps • What about dynamic content? • CDN now works for web apps, not just for public blobs • Same model as before, but source is your web app • Useful for stable dynamic content • Source URL: http://foo.cloudapp.net/default.aspx • CDN URL: http://azXXXX.vo.msecnd.net/default.aspx • Cached URL: http://foo.cloudapp.net/cdn/default.aspx • CNAMEs & HTTPS supported • Choose to index with or without query string • http://blog.smarx.com/posts/using-the-windows-azure-cdn-for-your-web-application

15. Windows Azure Traffic Manager • What about non-stable dynamic content, and writes? • Trick #4: Direct users to the service in the closest region with the Windows Azure Traffic Manager • CNAMEs supported • Useful for performance, business continuity, price, compliance & tax • Not the same as CDN • Not serving from the edge • Only DNS is cached (at client) foo-us.cloudapp.net Traffic Manager foo.cloudapp.net foo-europe.cloudapp.net DNS response Monitoring Policies 1.2.3.4 foo-asia.cloudapp.net

16. Traffic Manager Details • Multiple factors determine DNS resolution • Configured by Microsoft • Geo-IP mapping • Periodic performance measurement • Configured by service owner • Policy: Performance, Failover, Geo, Ratio • Monitoring • Currently in CTP • Only works in “production” slot but not yet intended for production use • No SLA or billing, and the Traffic Manager domain name will change foo-us.cloudapp.net foo-europe.cloudapp.net Periodic GETs Monitoring /monitoring/testme.aspx foo-asia.cloudapp.net

17. Part 2(b): Managing Server Data Access

18. In-Memory Caching • Trick #5: Cache hot data in memory to avoid data-tier access • Session state (e.g. shopping cart) & immutable reference data (e.g. product catalog entries) • Caching tier will help you reduce latency and cost • Lower latency/higher throughput than data tier, especially under load In-Memory Caching SQL Azure SQL Azure Hosted Compute Table Storage Table Storage

19. Windows Azure AppFabric Caching • What is it? • AppFabric Caching is a “Distributed In-Memory Cache As A Service” • Similar to on-premises Windows Server AppFabric Cache • Roadmap to full parity between on-premise and cloud where it makes sense • Excellent performance • Highly scalable, 64-bit service • Low latency, hosted per subregion for app affinity • Highly-available • AuthN/AuthZ integrated with Access Control Service • You may have heard of memcached too • Latest memcached works on Windows Azure with 1.3+ SDK • http://code.msdn.microsoft.com/winazurememcached • You manage instances and pay for compute hours

20. AppFabric Caching Advantages • Simple to administer • No need to manage and host a distributed cache yourself • Integrates easily into existing applications • ASP.NET session state and output cache providers enable no-code integration • Same managed interfaces as Windows Server AppFabric Cache • Near cache (client-local) for hot data without serialization costs Windows Azure App On-Premises App Windows Server AppFabric Cache Windows Azure AppFabric Caching Core Logic AppFabric Cache APIs Core Logic AppFabric Cache APIs

21. SQL Azure & Sharding • Trick #6: Partition (or shard) your SQL Azure data across databases • Spreads load across multiple database instances • Avoid hitting database size limits • Parallelized queries across more nodes • Improved query performance on commodity hardware • Partitioning scheme varies per data set • My sample scheme is not a good one! Why not? A-M A-Z Hosted Compute N-Z

22. Part 3: Performance Tuning

23. Basic Performance Tuning • Tune Windows Azure applications just as you would on-premises applications • Measure & optimize where it makes a difference • Windows Azure uses Full IIS for web roles in the 1.3+ SDK • Startup admin tasks using WebPI can install up-to-date extensions as desired • Startup admin tasks using AppCmd can configure IIS as desired • Basic tips • Build in release mode (not debug mode) for production • Do not enable IntelliTrace or Failed Request Tracing in production • Tune role instance counts and consider dynamic scaling

24. Advanced Performance Tuning • Enable compression for additional dynamic content types <add mimeType="application/json" enabled="true" /> <add mimeType="application/json; charset=utf-8" enabled="true" /> • Office document formats • Tune application pool recycling to suit your workload • Modify default schedule to avoid peak times • http://blog.smarx.com/posts/controlling-application-pool-idle-timeouts-in-windows-azure • Measure and eliminate memory leaks if footprint grows over time • Move ASP.NET cache to the resource disk for more space • Tune Windows Azure Diagnostics settings

25. Summary Blob Storage Cache control Versioned URLs CDN • Approach WA apps like you would on-premises apps • Use rich platform features in Windows Azure to tune for the cloud too Public Public Web App Private Traffic Mgr Asynchronous Hosted Compute Asynchronous Hosted Compute AppFabric Caching Shared Access Signatures Table Storage Table Storage Synchronous Hosted Compute Stg Key Tuning SQL Azure SQL Azure SQL Azure Sharding

26. Required Slide Speakers, please list the Breakout Sessions, Interactive Discussions, Labs, Demo Stations and Certification Exam that relate to your session. Also indicate when they can find you staffing in the TLC. Related Content • BOF10-DEV Developing for Windows Azure: Real-World Lessons from Real-World Projects • BOF12-DEV How Do I Design Scalable Cloud Applications • COS301 Inside Windows Azure, the Cloud Operating System with Mark Russinovich • COS318 A Lap Around Windows Azure AppFabric COS276-INT Windows Azure Traffic Manager: Improve Application Performance and Availability Using Global Load Balancing • COS275-HOL Windows Azure Content Distribution Network (CDN) • CS285-HOL Microsoft SQL Azure Tips and Tricks • Find Me Later At the Windows Azure booth or Cloud Power Lounge

27. Track Resources • Don’t forget to visit the Cloud Power area within the TLC (Blue Section) to see product demos and speak with experts about the Server & Cloud Platform solutions that help drive your business forward. • You can also find the latest information about our products at the following links: • Cloud Power - http://www.microsoft.com/cloud/ • Private Cloud - http://www.microsoft.com/privatecloud/ • Windows Server - http://www.microsoft.com/windowsserver/ • Windows Azure - http://www.microsoft.com/windowsazure/ • Microsoft System Center - http://www.microsoft.com/systemcenter/ • Microsoft Forefront - http://www.microsoft.com/forefront/

28. Resources • Connect. Share. Discuss. http://northamerica.msteched.com Learning • Sessions On-Demand & Community • Microsoft Certification & Training Resources www.microsoft.com/teched www.microsoft.com/learning • Resources for IT Professionals • Resources for Developers http://microsoft.com/technet http://microsoft.com/msdn

29. Complete an evaluation on CommNet and enter to win!

31. © 2011 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.