Timecard : Controlling User-Perceived Delays in Server-Based Mobile Applications

1. Timecard:Controlling User-Perceived Delays in Server-Based Mobile Applications Lenin Ravindranath, Jitu Padhye, Ratul Mahajan, Hari Balakrishnan

2. Cloud Services Mobile Apps

3. Response Time Matters Cloud Services • Users are impatient • 100ms delay can cost substantial drop in revenue at Amazon • Similar observations from Google and Bing

4. Control Variability in Response Times Deadline • Fixed deadlines • Trade-off quality for response time • More time to compute, better quality results • Flexible Services • For e.g., search Server processing Request Response

5. The elephant is outside the room Server processing Request Response

6. The elephant is outside the room Server processing Request Response Server Phone model Uplink Downlink Reading sensors Response size Link quality (3G, HSPA+, LTE, Wifi) App App Radio State User click Parsing and Rendering Highly variable User perceived delay

7. The elephant is outside the room • Unaware of end-to-end delays • Clients with non-trivial external delays • Poor end-to-end response times • Client with low external delays • Do not produce the best quality result Server processing Request Response Server Phone model Uplink Downlink Reading sensors Response size Link quality (3G, HSPA+, LTE, Wifi) App App Radio State User click Parsing and Rendering Highly variable User perceived delay

8. The elephant is outside the room Server processing Request Response Server Phone model Uplink Downlink Servers should adapt to external delays and control end-to-end delay variability Reading sensors Response size Link quality (3G, HSPA+, LTE, Wifi) App App Radio State User click Parsing and Rendering Highly variable User perceived delay

9. Timecard PredictRemainingTime (responseSize); GetElapsedTime(); Server App App Predicted downlink + app processing delay Time elapsed since user request

10. Timecard PredictRemainingTime (responseSize); GetElapsedTime(); Adapt Processing Time Server App App App Desired end-to-end delay

11. Timecard PredictRemainingTime (responseSize); GetElapsedTime(); Server Adapt Response App App App Desired end-to-end delay

12. Timecard PredictRemainingTime (responseSize); GetElapsedTime(); Server App App Desired end-to-end delay

13. Timecard PredictRemainingTime (responseSize); GetElapsedTime(); Server App App

14. Challenges Send response Spawn workers Request handler Server Threads Server App App Background Thread GPS callback Web request Web callback Background Thread UI dispatcher GPS start Event handler UI Thread Thread start

15. Challenges GetElapsedTime(); PredictRemainingTime (responseSize); No single reference clock Automatically collect data and learn Server App App Background Thread Background Thread UI Thread Transaction

16. Timecard • Online Transaction Tracking • Time Synchronization • Remaining Time Predictor • Downlink delay • App processing delay

17. Synchronizing Time • Periodic time sync probes from app to server • Find drift and offset between clocks • Use server clock as reference clock • Client maps local time to server time • Efficient technique for probing • Aware of the radio state and traffic • Minimal extra delays • Energy efficient

18. Predicting Remaining Time PredictRemainingTime (responseSize); • Downlink time • App processing time Server App App

19. Predicting Downlink Time Read TCP state • 1 KB to 40 KB of data • RTT matters than throughput • Predict RTT • TCP window state matters • Multiple RTTs • Estimate TCP Window & number of RTTs • Complicated by middleboxes Data size Middlebox Estimate TCP state

20. Predicting Downlink Time • Model downlink delay • Recent RTT • Bytes already transferred • Response size • Network provider • Client OS Data size Middlebox Estimate TCP state • Error in cellular network • median 17 ms • 90th percentile 86ms

21. Predicting App Processing Time • Parsing and rendering depends on data size • Model processing time as a function of • Response data size • Phone model 4.6% (8ms) median error, 10% in 90th percentile

22. Timecard Implementation Data Collection Automatic Instrumentation • WP Apps • ASP .NET services APIs Predictor Server Transaction Tracking App App Logging Transaction Tracking Time Sync

23. Timecard can help control end-to-end delay • Modified two services (and two apps) to use timecard Mobile Ads Service

24. Timecard Overhead • 0.1% runtime overhead • Less than 1% memory overhead • Garbage collection in idle time • Less than 1% network overhead • Negligible battery overhead

25. Timecard PredictRemainingTime (responseSize); GetElapsedTime(); Adapt Processing Time Server Adapt Response App App Desired end-to-end delay

26. Timecard PredictRemainingTime (responseSize); GetElapsedTime(); Request Prioritization Server App App Desired end-to-end delay

27. Timecard PredictRemainingTime (responseSize); GetElapsedTime(); Adapt Resources Used Server App App Desired end-to-end delay

28. Backup

29. Transaction Tracking GetTransaction(). GetStartTime(); TC TC Server Threads Server TC App App TC Background Thread TC Background Thread TC TC UI Thread TC Transaction Context TC - Timestamps, transaction/device/network information