A Feedback Control Architecture and Design Methodology for Service Delay Guarantees in Web Servers

1. A Feedback Control Architecture and Design Methodology for Service DelayGuarantees in Web Servers Presentation by Amitayu Das

2. Introduction • Response-time delay while browsing • Implications • Loss for the website • Loss of revenue for the hosting platform, too • Reasons • Server end problem • Network latency We’re talking about Server end problem

3. Motivation • Time-varying workload for limited resource • Limited adaptation of web-server: best-effort • Lack of enforcement of QoS guarantees at web-server • Notion of service differentiation is not enforced at server end

4. Differential service • Two classes: premium, basic (say) • Best-effort model: no guarantee • Absolute model: • soft deadline • How to decide the deadline? Depends on several things • No overload => all classes receive satisfactory delay • Overload => degradation in prioritized order • Proportional model: no fixed deadline, hence flexible • Performance differentiation is better than previous two • Hybrid model: • Gets the best of above two • Flexibility with no overload, bounded delay for high priority classes on overload

5. Web server mechanism • Scenario for web server • Handle incoming TCP connection by assigning a server • Multi-threaded/multi-process setup • Multi-threaded setup is very costly in UNIX • HTTP 1.0: • Excessive # of concurrent TCP connection • HTTP 1.1: • Persistent connection and problems with that • Which is the bottleneck here?

6. Service delay guarantees • Connection delay: • Time b/w arrival and acceptance • Processing delay: • Time b/w arrival and transferring response to client • Connection delay (Ck(m)): average for class k (0  k  N) within ((m-1)S, mS) • Relative delay guarantee: Cj(m)/Cl(m) = Wj/Wl for all j and l (j ≠ l); Wj is the relative desired delay • Absolute delay guarantee: Cj(m)  Wj; for all classes j if there is a class l > j and Cl(m)  Wl , which is desired (absolute) delay • Hybrid delay guarantee: Wk represents both desired delay and relative delay

7. The Feedback-Control Architecture for Delay Guarantees

8. Delay controllers • Controller: (Reference, Output, Error, Control Input) (VSk, Vk(m), Ek(m), Uk(m)) • Absolute delay controller CAk: (Wk, Ck(m), VSK(m) – Vk(m), Bk(m)) • Relative delay controller CRk: (Wk/Wk-1, Ck(m)/Ck-1(m), VSK(m) – Vk(m), Bk-1(m)/Bk(m)) • Hybrid delay controller • Switching condition: • C0 (m) > W0 + H, switch to CA; H is a threshold, to avoid • C0 (m) > W0 - H, switch to CR; thrashing b/w controllers

9. Design of delay-controller • Performance specification • Stability • Settling time (TS):measures efficiency of controller • Steady state error (ES): measures accuracy • System Identification: establish dynamic model • Root Locus: designs controller to meet performance specification

10. Architecture for system identification • System identification • Model structure • White noise input • LSE • Estimated parameters • (a1, a2, b1, b2) = (0.74, -0.37, 0.95, -0.12): relative delay • (a1, a2, b1, b2) = (0.08, -0.2, 0.2, -0.05): absolute delay

11. System identification results for relative delay

12. Results (relative delay)

13. Results (absolute delay)

14. Root Locus design • g = 0.3, r = 0.05 for relative delay controller • g = -4.6, r = 0.3 for absolute delay controller

15. Evaluation of relative-delay guarantees

16. Evaluation of relative-delay guarantees

17. Evaluation of absolute-delay guarantees

18. What self-* about it? • Proposes adaptive architecture • Avoids laborious ad-hoc approaches for tuning and design iteration

19. Result with three classes

20. Last slide • Questions??