Guarantees and Protocols: Abstractions for Recovery

1. Guarantees and Protocols: Abstractions for Recovery Krithi Ramamritham, IIT Bombay (with Cris Pedregal-Martin)

2. transactions and recovery review Transactions: all-or-nothing semantics • if transaction commits, all updates made permanent • if aborts, no updates ever visible to others a.k.a. Failure Atomicity and Durability (FA+D) Recovery supports all-or-nothing • uncommitted updates undoable (never happened) • once committed, database reflects updates in spite of failures

3. transactions are everywhere! applications: • e-commerce, workflows • mobile systems • design, document management, etc. want variants of transaction semantics • tailored to application, platform • to get benefits of recovery, concurrency

4. we focus on Recovery liveness wrt failures: Recovery enables a system to make progress in spite of (temporary) failures we assume simple temporary failures e.g., loss of volatile memory, communication

5. outline for talk • recovery beyond databases very desirable • …but hard to get right • our approach & contributions • e-commerce • mobile system • related work • summary and future work

6. recovery is hard to get right • cuts through all system levels • failure functionality - performance intertwined e.g., fine control over storage management • recovery is custom-made • very good database solutions (e.g. ARIES) • but, hard to (re) deploy, adapt: change infrastructure, effect on high-level semantics? change high-level, what’s required of infrastructure? no easy answers

7. example: delegation high-level primitive for advanced transactions e.g. nested, split trans. obtained with delegation T1 delegates operation p to T2 p’s fate is now tied to T2’s • e.g. T2 commits then p appears even if T1 aborted delegation “rewrites history” (p by T2 not T1) to support it, modify low-level recovery nontrivial in e.g. ARIES! Delegation: Efficiently Rewriting History, in 13th ICDE, 1997

8. we want abstraction in recovery • goal: separate what from how of recovery • approach: • find common ingredients & their roles • formalize & use to describe recovery • implicit temporal predicates (liveness) • decompose / describe: • application-level semantics down to implementation-level mechanisms (vertical) • recovery obtained from autonomous components (horizontal)

9. contributions: framework & examples 1. framework to specify & reason about recovery • guarantees: recovery expectations of components e.g., Bank will pay authorized charge, even under failures • protocols: legal component behaviors e.g., authorize charge happens-before confirm order (Merchant) forced protocols: progress of application specifications separate what from how 2. applied to case studies • e-commerce • database recovery • mobile systems

10. outline for talk • recovery necessary but hard to get right • our approach & contributions • e-commerce • overview, framework, layered spec. & proofs • mobile system • related work • summary and future work

11. e-commerce timeline Customer begin-order Merchant open-order time

12. e-commerce timeline Customer begin-order Bank authOK Merchant auth? GBM open-order Bank promises Merchant it will pay charge time

13. e-commerce timeline Customer begin-order Bank authOK Merchant auth? GBM Merchant can commit order-OK open-order GSM allot? Supplier promises Merchant it will ship allotOK Supplier time

14. e-commerce timeline Customer order-received end-order begin-order pay-bank Bank bill-customer authOK pay-merchant Merchant auth? charge-bank GBM order-OK pay-supplier open-order GSM allot? order-shipment bill-merchant allotOK Supplier ship-goods time Merchant commits order to Customer

15. e-commerce timeline observations • money-goods atomicity ensured by • each component honors its guarantee and makes progress • each guarantee becomes enabled • once Merchant confirms order, inevitably each guarantee is triggered, so eventually discharged • discharge of each guarantee pays or delivers • no need to look inside the components

16. elements of the framework • system history and actions/events • partial order on data operations, commit, abort, ... • components - • subsystems that offer/use guarantees • guarantees (liveness) - promise future events • enable, trigger, discharge events/actions • components that request and that honor guarantee • protocols (safety) - prescribe legal histories • happens-before relationship between events • forced protocols (liveness) • operation sequences will make progress

17. partial layers e-commerce scenario order confirmed money exchanged for goods Bank pays Merchant & ... Merchant - Bank messages GBM enabled, triggered, discharged Bank Application uses Transactions Bank supports GBM to Merchant Database supports commit, abort Bank’s DB Transactions Guarantees Buffer Mgmt, Disk persistence Database Recovery support

18. money for goods M C orderOK H  Supplier ships goods to Customer Merchant pays Supplier Bank pays Merchant Customer pays Bank & & &

19. money for goods M C orderOK H  Bank pays Merchant GBM discharged Supplier ships goods to Customer Merchant pays Supplier Customer pays Bank & & & GBM enabled GBM triggered Bank’s internals support GBM

20. MP1: Merchant’s workflow engine (scheduler) money for goods M C orderOK H  Bank pays Merchant GBM discharged Supplier ships goods to Customer Merchant pays Supplier Customer pays Bank & & & GBM enabled B M authOK H GBM triggered M B charge H Bank’s internals support GBM orderOK H  MP1: authOK  orderOK

21. MP1: Merchant’s workflow engine (scheduler) money for goods M C orderOK H  Bank pays Merchant GBM discharged Supplier ships goods to Customer Merchant pays Supplier Customer pays Bank & & & GBM enabled B M authOK H GBM triggered M B charge H Bank’s internals support GBM Xauth, Xpay transactions orderOK H  MP1: authOK  orderOK

22. MP1: Merchant’s workflow engine (scheduler) money for goods M C orderOK H  Bank pays Merchant GBM discharged Supplier ships goods to Customer Merchant pays Supplier Customer pays Bank & & & GBM enabled B M authOK H GBM triggered M B charge H Bank’s internals support GBM Xauth, Xpay transactions orderOK H  MP1: authOK  orderOK orderOK H  MFP4: orderOK  charge Database TMS

23. MP1: Merchant’s workflow engine (scheduler) MFP4: Merchant’s workflow engine (recovery) money for goods M C orderOK H  Bank pays Merchant GBM discharged Supplier ships goods to Customer Merchant pays Supplier Customer pays Bank & & & GBM enabled B M authOK H GBM triggered M B charge H Bank’s internals support GBM Xauth, Xpay transactions orderOK H  MP1: authOK  orderOK orderOK H  MFP4: orderOK  charge Database TMS Disk

24. prove one:order confirmed  merchant paid (S, C, shipment: x) (C, B, payment: p) Merchant Customer Bank Supplier (B, M, payment: p) (M, S, payment: p)

25. how order confirmed  merchant paid Prove MC orderOK  H  BM payment  H

26. how order confirmed  merchant paid Prove MC orderOK  H  BM payment  H Hint: BM payment = dischargeGBM (Bank will pay auth. charges) Prove: GBM supported, enabled, and triggered

27. how order confirmed  merchant paid Prove MC orderOK  H  BM payment  H Hint: BM payment = dischargeGBM (Bank will pay auth. charges) Prove: GBM supported, enabled, and triggered • Bank supports GBM - given by its database

28. how order confirmed  merchant paid Prove MC orderOK  H  BM payment  H Hint: BM payment = dischargeGBM (Bank will pay auth. charges) Prove: GBM supported, enabled, and triggered • Bank supports GBM - given by its database • enableGBM  H via: MC orderOK  H  BM authOK  H • protocol P1: BM authOK  MC orderOK

29. how order confirmed  merchant paid Prove MC orderOK  H  BM payment  H Hint: BM payment = dischargeGBM (Bank will pay auth. charges) Prove: GBM supported, enabled, and triggered • Bank supports GBM - given by its database • enableGBM  H via: MC orderOK  H  BM authOK  H • protocol P1: BM authOK  MC orderOK • triggerGBM  H via: MC orderOK  H  MB payreq  H • Merchant forced protocol P5 - by its application engine • pre(MB payreq) = (enableGBM  H  MC orderOK  H)

30. how Bank supports guarantee Merchant GBM: Bank will pay Merchant authorized charges, even under failures: • transaction Xauth commits authorization data • authID will be recognized when presented • transaction Xpayout responds to charge request • finds authorization info via authID, commits payment • each Merchant request ack’d at commit Merchant will charge, by its forced protocol Database Transaction semantics given

31. e-commerce observations • transactional semantics • M C Order OK is like transaction commit • recovery support ensures sale completes • abstraction: • prove properties one layer at a time • can spec. component w/out others internals • easy to specify alternatives • Merchant self-authorizes under some amount • Merchant escrows goods at Supplier

32. outline for talk • recovery necessary but hard to get right • our approach & contributions • e-commerce • mobile system • overview, challenges, specification, observations • related work • summary and future work

33. M mobile mobile system handoff A base station B server S fixed network

34. M mobile mobile system handoff A base station B server S fixed network

35. mobile system challenges • handoff dynamically reconfigures network • mobile doze/suspend partitions network • limited storage on mobile host, base station problem: • preserve recovery for mobile when it migrates • operations done while M at A • commit / recovery when M at B solution: handoffs do some recovery work

36. handoff variants • propagation of recovery information • eager, during normal processing • lazy, during repair processing • directly or via another mobile host • location of persistent recovery information • at base stations • at central server for the different variants, support recoverability

37. mobile system layers

38. mobile system observations • characterized variants & showed how recovery supported regardless of variant • exposed unstated assumptions about handoffs when proving recovery properties • proved how higher-level obtained from lower level guarantees

39. related work • ARIES: database recovery, specification • Sagas, ACTA: advanced transaction models • Phoenix: recovery for office application • ProMotion: mobile transactions

40. contributions • framework: • identified & formalized common ingredients • used ingredients to separate what from how used framework to reason about examples: • e-commerce • “liveness” from autonomous parts Guaranteeing Recoverability in E-Commerce, in 3rd WECWIS, 2001 • mobile systems • exposed assumptions, novel scenarios... Recovery Guarantees in Mobile Systems, in MobiDE, 1999 Support for Recovery in Mobile Systems, in IEEE Trans on Computers, Oct, 2002

41. future work extend framework • examples: ARIES variants, client-server, apps. • add quantitative extensions • recovery as part of quality of service (levels?) • semiautomatic tool / toolkit BTW: abstraction is no panacea! [Massalin, Shrikumar] other directions • data-based applications as utility services • Heterogeneity, Autonomy, Distribution -- Robustness? • “Quality of Data”/ “Data Proxies”

42. related work Transactions and Recovery theory and practice: • Bernstein et al., Wallace et al., Kuo (ARIES) • Gray and Reuter, Cabrera et al., Mohan et al. Advanced Transaction Models • Chrysanthis and Ramamritham (ACTA) • García-Molina (Sagas) • Elmagarmid (ed.); Jajodia and Kerschberg (eds.)

43. related work (ctd.) Electronic Commerce • Tygar, Schuldt et al. Recovery beyond databases • Lomet et al. (applications) • Kamath, Casati et al. (workflows) • Alonso and Korth, Barbará (mobile sys. surveys) • Chrysanthis et al., Pradhan et al., Madria and Bhargava, Yao et al. (mobile systems)