modeling and analyzing fault tolerant real time communication protocols n.
Skip this Video
Loading SlideShow in 5 Seconds..
Modeling and Analyzing Fault-Tolerant, Real-Time Communication Protocols PowerPoint Presentation
Download Presentation
Modeling and Analyzing Fault-Tolerant, Real-Time Communication Protocols

Modeling and Analyzing Fault-Tolerant, Real-Time Communication Protocols

133 Vues Download Presentation
Télécharger la présentation

Modeling and Analyzing Fault-Tolerant, Real-Time Communication Protocols

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. Modeling and Analyzing Fault-Tolerant, Real-Time Communication Protocols Nancy Lynch Theory of Distributed Systems MIT Second MURI Workshop Berkeley, California June 4, 2001

  2. MIT Participants • Leaders: Nancy Lynch, Idit Keidar • Students: Carl Livadas, Roger Khazan, Ziv Bar-Joseph • Collaborators: Paul Attie, Alex Shvartsman, Roberto Segala, Frits Vaandrager

  3. At Last Year’s Workshop…

  4. General Models and Proof Methods • I/O automaton models [Lynch, Tuttle 87] • Nondeterministic, infinite-state machines • Input/output/internal actions, traces • Modularity: Composition, levels of abstraction • Mathematical, language-independent • Used to model distributed algorithms, communication protocols • Validation, code generation, upper and lower bounds

  5. Timing, Hybrid Considerations • Timing: TIOAs [Lynch, Vaandrager] • Timeout-based algorithms. • Local clocks, clock synchronization • Performance analysis • Hybrid: HIOAs [L, Segala, V, Weinberg 96] • Real world + computer components • Continuous flows of data

  6. Other Embellishments • Probabilities: PIOA, PTIOA [Segala 95] • Probabilistic and nondeterministic behavior. • Randomized distributed algorithms • Systems with probabilistic assumptions • Dynamic systems: DIOA [Attie, Lynch 99] • Run-time process creation and destruction, mobility. • Agent systems

  7. Communication Protocol Modeling and Analysis. • At-most-once (AMO) Message Delivery • TCP, T/TCP • Reliable channels from unreliable channels • Self-stabilizing communication protocols • Network clock synchronization • Group communication systems

  8. Group Communication Service • Communication middleware • Manages group membership, current view • Handles joins, leaves, failures, partitions, merges • Multicast communication among members • Multicasts respect views • Ordering, reliability constraints for message delivery, e.g., FIFO, causal within each view. • Isis, Transis, Totem,…

  9. VStoTO brcv bcast TO VStoTO VStoTO gprcv newview gpsnd VS

  10. Conditional Performance Analysis • Assume VS satisfies: • If a network component C stabilizes, then soon thereafter, views become consistent within C, and messages sent in the final view are delivered everywhere in C, within bounded time. • And VStoTO satisfies: • Simple timing, fault-tolerance assumptions. • Then TO satisfies: • If C stabilizes, then soon thereafter, any message sent or delivered anywhere in C is delivered everywhere in C, within bounded time.

  11. Conditional Performance Analysis • Give conditional claims about system performance under particular assumptions about behavior of environment and of network substrate, e.g.: • Stabilization of underlying network. • Limited rate of change. • Bounds on message delay. • Limited amount of failure (number, density). • Limited input arrivals (number, density). • Assumptions => Guarantees. • Get probabilistic statements as corollaries. • Composable

  12. What we proposed: 1. Model, analyze communication protocols. 2. Develop conditional performance analysis techniques. 3. Extend I/O automata theory to accommodate performance, reliability, hybrid, probability, dynamic considerations. 4. Relate, integrate I/O automata with other frameworks.

  13. Progress this year 1. Communication protocol design/analysis • Scalable Group Communication • Totally Ordered Multicast with QoS • Scalable Reliable Multicast • Conditional performance analysis methods • Evolving… 3. I/O automaton models • Hybrid I/O Automata • Dynamic I/O Automata • IOA language support 4. Comparing, integrating with other models • A start…

  14. Protocol Modeling/Analysis

  15. Scalable Group Communication[Keidar, Khazan 00]

  16. Group Communication Service • Manages group membership, current view. • Multicast communication among group members, with ordering, reliability guarantees. • Virtual Synchrony[Birman, Joseph 87] • Integrates group membership and group communication. • Processes that move together from one view to another deliver the same messages in the first view. • Useful for replicated data management. • Before announcing new view, processes must synchronize, exchange messages.

  17. Example: Virtual Synchrony i j k 3: i,j,k 3: i,j,k 3: i,j,k mcast(m) rcv(m) rcv(m) 4: i, j 4: i, j VS algorithm supplies missing m

  18. VSGC Net GM Group Communication in WANs • Difficulties: • High message latency, message exchanges are expensive • Frequent connectivity changes • New, scalable GC algorithm: • Uses scalable GM service of [Keidar, Sussman, et al. 00], implemented on special membership servers. • GC (with virtual synchrony) implemented on clients. VS

  19. Group Communication in WANs • Try to minimize time from when network stabilizes until GC delivers new views to clients. • After stabilization: GM forms view, VSGC algorithm synchronizes. • Existing systems (LANs): • GM, VSGC uses several message exchange rounds • Continue in spite of new network events • Inappropriate for WANs view(v) Net event VSGC Algorithm GM Algorithm

  20. view(v) New Algorithm • VSGC uses one message exchange round, in parallel with GM’s agreement on views. • GM usually delivers views in one message exchange. • Responds to new network events during reconfiguration: • GM produces new membership sets • VSGC responds to membership changes • Distributed implementation [Tarashchanskiy 00] Net event VSGC Algorithm GM Algorithm

  21. S S’ A A’ Correctness Proofs • Models, proofs (safety and liveness) • Developed new incremental modeling, proof methods [Keidar, Khazan, Lynch, Shvartsman 00] • Proof Extension Theorem: • Used new methods for the safety proofs.

  22. Performance Analysis • Analyze time from when network stabilizes until GC delivers new views to clients. • Compare with other strategies. • System is a composition: • Network and GM services, plus • VSGC processes • Use composition in the analysis.

  23. Performance Analysis • Analyze the VSGC algorithm alone, in terms of its inputs and timing assumptions. • State reasonable performance guarantees for GM and Network. • Combine to get conditional performance properties for the system as a whole.

  24. 1. Analysis of VSGC algorithm • Assume component C stabilizes: • GM delivers same views • Net provides reliable communication with latency . • Let • T[start], T[view] be times of last GM events for C •  be upper bound on local step time. • Then VSGC outputs new views by time max (T[start] +  + x, T[view]) + 

  25. view(v) view(v)  + x Analysis of VSGC Algorithm VS Algorithm Net Event start start GM algorithm T[start] T[view]

  26. 2. Assumed Bounds for GM T[start] T[view] • Bounds for “Fast Path” of [Keidar, et al. 00],observed empirically in almost all cases. start  start view(v)  GM

  27. view(v) 3. Combining VSGC and GM Bounds  + x • Bounds for system, conditional on GM bounds. VSGC start start view(v)   T[start] T[view] GM

  28. Totally Ordered Multicast with QoS [Bar-Joseph, Keidar, Anker, Lynch 00]

  29. Totally Ordered Multicast with QoS • Multicast to dynamic group, subject to joins, leaves, and failures. • Global total ordering of messages • QoS: Message delivery latency • Built on reliable network with latency guarantees • Add ordering guarantees, preserve latency bounds. • Target applications • State machine replication • Military command and control • Distributed games • Shared editing

  30. Two Algorithms • Algorithm 1: Basic Totally Ordered Multicast • Sends, receives consistent with total ordering of messages. • Non-failing processes agree on messages from non-failing processes. • Latency: Constant, even with joins, leaves, failures. • Algorithm 2: Atomic Multicast • Non-failing processes agree on all messages. • Latency: • Joins, leaves only: Constant • With failures: Linear in f TOM fail_i fail_j Net

  31. Local Node Process join leave rcv(m) mcast(m) Ord_i joiners(s,J), leavers(s,J) end-slot(s) FrontEnd_i members(s,J) Memb_i mcast(m) join leave mcast(join) mcast(leave) progress(s,j) Sniffer_i rcv(join) rcv(leave) rcv(m) Net

  32. Local Algorithm Operation • FrontEnd divides time into slots, tags messages with slots. • Ord delivers messages by slot, in order of process indices. • Memb determines slot membership. • Join, leave messages • Failures: • Algorithm 1 uses local failure detector. • Algorithm 2 uses consensus on failures. • Requires new dynamic version of consensus. • Timing-dependent

  33. Net GM Architecture for Algorithm 2 TO-QoS

  34. Performance Analysis (Planned) 1. Latency of TO-QoS in terms of GM 2. GM latency bounds 3. Combine

  35. Using Caching to Improve Reliable Multicast Algorithms[Livadas]

  36. SRM[Floyd, et al.] • Reliable multicast to dynamic group. • Built over IP multicast • Based on requests (NACKs) and retransmissions • Limits duplicate requests/replies using: • Deterministic suppression: Ancestors suppress descendants, by scheduling requests/replies based on distance to source. • Probabilistic suppression: Siblings suppress each other, by spreading out requests/replies.

  37. SRM Architecture SRM IPMcast

  38. New Protocol • Tries to improve SRM by using loss history information. • Useful if future losses occur on same link. • Uses deterministic suppression for siblings also • Determines, caches best requestor, best replier • Chooses requestor closest to source. • Chooses replier closest to requestor. • Break ties with processor ids. • Defaults to SRM Replier Requestor

  39. Performance • Metrics: • Loss recovery latency: Time from detection of packet loss to receipt of first retransmission • Loss recovery overhead: Number of messages multicast to recover from a message loss • Protocol performance benefits: • Removes delays caused by probabilistic suppression • Following election of requestor and replier: • Reduces latency by using best requestor and replier. • Reduces overhead by using single requestor and replier. • Latency analysis (Planned)

  40. 3. I/O Automaton Models

  41. Hybrid I/O Automata[Lynch, Segala, Vaandrager, HSCC 01] • New, simpler version of HIOA model of [LSVW96] • Supports decomposing hybrid system descriptions: • External behavior: Discrete actions and continuous flows • Composition: Synchronizes external actions and flows, respects external behavior • Abstraction: Implementation and simulation relation notions, respect external behavior. • Separate mechanisms: • External actions for discrete communication. • External variables for continuous flow.

  42. Del(d1) Del(d1) Del(d2) Del(d2) Example: Delay Buffer Del(d) • Accepts discrete and continuous input, produces isomorphic output, with delay d. • Compose in sequence, in cycle: • Composition implements Del(d1 + d2): Del(d1) Del(d2) Del(d1 + d2)

  43. Vehicle Sensor Actuator Controller Example: Vehicle and Controller • Keep vehicle speed in [v1, v2]. • Sensor senses velocity, reports to Controller every time d. • Controller suggests acceleration. • Vehicle follows suggested acceleration, with uncertainty ε. • Compose: Discrete, continuous interactions • Prove invariant: velocity in [v1,v2]. • Use auxiliary invariants, including timing. acc-in vel-out report(v) suggest(a)

  44. HOIA definition • U, X, Y: Input, output, internal (state) variables • Θ: Initial states • I, O, H: Input, output, internal actions • D, discrete transitions • T, trajectories • Mappings from time intervals to valuations of variables • Closure properties • Input-enabling for actions, flows • Execution:τ0, a1, τ1, a2, τ2, … • Trace: Restrict to external variables and actions

  45. Composition and Abstraction • Abstraction: • A implements B if comparable and traces(A) subset of traces(B). • Simulation relation: Start, step, trajectory conditions • Theorem: Simulation relation implies implementation • Composition: • Synchronize external actions and variables • Theorems: Projection, pasting, substitutivity • Receptiveness: • Doesn’t cooperative in producing Zeno behavior • Theorem: Closed under composition (with technical assumption).

  46. Dynamic I/O Automata [Attie, Lynch, Concur 01] • Dynamic version of I/O automata, including: • Automaton creation and destruction • Signature change • Two-level model: Automata, configurations. • Mobility modeled using signature change

  47. I A O IOA Language and Tools • Language for describing I/O automata: [Garland, Lynch] • Front end:[Garland] • Translates to Java objects • Completely rewritten this year. • Needs support for composition. • Theorem-prover connection: [Garland, Bogdanov] • Connection with LP • Seeking connections: SAL, Isabelle, STeP, NuPRL

  48. IOA Language and Tools • Simulator:[Chefter, Ramirez, Dean] • Has support for paired simulation. • Needs additions. • Being instrumented for invariant discovery using Daikon [Ernst] • Code generator:Tauber, Tsai • Local code-gen (translation to Java) running. • Needs composition, communication service calls, correctness proof. • Challenge examples

  49. Plans

  50. Plans 1. Protocol modeling/verification • Finish analysis of Scalable GC, Totally Ordered Multicast with QoS, SRM • Other protocols from this project. 2. Conditional analysis methods • Develop general methods • Compare with other methods (Trivedi)