1. Consistency and Replication • The problems we are trying to solve • Types of consistency • Approaches to propagation Distributed Systems - Comp 655

2. Transparency in a Distributed System Distributed Systems - Comp 655

3. What problems does replication solve? • Some capacity and performance problems • Keep replicas on both sides of a bottleneck • Keep replicas on both sides of a connection with long delays • Two kinds of incoherence: • Replication provides some location transparency • Replication provides some failure transparency (aka fault tolerance) • Continue to work if one copy goes down • Continue to work if the network goes down Distributed Systems - Comp 655

4. What problems does replication cause? • Consistency • To maintain concurrency transparency, system has to keep replicas updated • Complexity • To maintain replication transparency, system has to be able to locate and select appropriate replicas • Overhead can take back capacity and performance gains Distributed Systems - Comp 655

5. If you remember onlyone two thing(s) … • There are many types of consistency, known as “consistency models” • As the consistency model gets stronger • The system gets easier to use. • The system gets harder to implement. • The system gets slower and consumes more resources. Distributed Systems - Comp 655

6. Consistency and Replication • The problems we are trying to solve • Types of consistency • Approaches to propagation Distributed Systems - Comp 655

7. Types of (data-centric) consistency Distributed Systems - Comp 655

8. Ground rules for examples • Examples deal with a company’s inventory data • Inventory data is replicated at every location • This is for illustrative purposes; we are NOT claiming this is a good architectural idea • Assume each process in each example is at a different location • Assume that invisible processes propagate each update to all other copies • Time moves left to right Distributed Systems - Comp 655

9. Strict consistency D=H P1: P2: D?H Yes P1: D=H P2: D?L D?H No D:DVD players, H:high, L:low, set:=, get:? Distributed Systems - Comp 655

10. Sequential consistency 1 D=H P1: P2: D=L P3: D?L D?L D?L D?L P4: D:DVD players, H:high, L:low, set:=, get:? Distributed Systems - Comp 655

11. Sequential consistency 2 D=H P1: P2: D=L P3: D?L D?H P4: D?L D?H D:DVD players, H:high, L:low, set:=, get:? Distributed Systems - Comp 655

12. Sequential consistency - not D=H P1: P2: D=L P3: D?L D?H P4: D?H D?L D:DVD players, H:high, L:low, set:=, get:? Distributed Systems - Comp 655

13. Cost of sequential consistency • Assume we have a distributed system with a replicated store and a feature to maintain consistency across the replicas • Let r be the expected read time • Let w be the expected write time • Let t be the minimal packet transfer time between nodes in the system • Then r + w  t Distributed Systems - Comp 655

14. Potential causal relationships No causal relationship Not sequentially consistent Causal consistency P1: D=L D=M P2: D?L D=H D?L D?M D?H P3: P4: D?L D?H D?M D:DVD players, H:high, M: medium, L:low, set:=, get:? Distributed Systems - Comp 655

15. Potential causal relationship Causal consistency - not D=L P1: P2: D?L D=H D?L D?H P3: D?H D?L P4: D:DVD players, H:high, L:low, set:=, get:? Distributed Systems - Comp 655

16. No causal relationship Causal consistency - ok D=L P1: P2: D=M D?L D?M P3: D?M D?L P4: D:DVD players, M:medium, L:low, set:=, get:? Distributed Systems - Comp 655

17. Types of (data-centric) consistency    Distributed Systems - Comp 655

18. Eventual consistency • DNS • A domain can only be updated by its naming authority (therefore, no write-write conflicts) • Updates propagated on a schedule • Most accesses work from cache • Can refresh a cached value on failure • Web • A page can only be updated by its owner (again, no write-write conflicts) • Most accesses work from cache • Page can be refreshed if user suspects a failure Distributed Systems - Comp 655

19. Client-centric consistency Distributed Systems - Comp 655

20. Client-centric consistency models Distributed Systems - Comp 655

21. Client-centric examples • Imagine a discussion group with replicas in Columbus and Nairobi • Imagine these events • Alice starts thread A • Bob starts thread B • Alice edits A to make A’ • Alice adds attachment to make A’’ • Carol posts reply AC Distributed Systems - Comp 655

22. Not mono read OK Monotonic reads Columbus Nairobi A A B B Carol reads Carol reads again Distributed Systems - Comp 655

23. Not mono read OK Monotonic reads Columbus Nairobi A B B A Carol reads Carol reads again Distributed Systems - Comp 655

24. Not mono write OK Monotonic writes Columbus Nairobi A A A’ A’ A’’ Distributed Systems - Comp 655

25. Not read your writes Read your writes, but Non-monotonic reads OK Read your writes Columbus Nairobi A B B A AC AC Alice reads Alice reads again Distributed Systems - Comp 655

26. Not writes follow reads OK Writes follow reads Columbus Nairobi A Carol reads A AC AC arrives Distributed Systems - Comp 655

27. Dave sees reply before original Writes follow reads to prevent this Columbus Nairobi A Carol reads AC AC Bob reads Dave reads A Distributed Systems - Comp 655

28. Consistency and Replication • The problems we are trying to solve • Types of consistency • Approaches to propagation Distributed Systems - Comp 655

29. Types of replicas Any experience with server-initiated replicas? Distributed Systems - Comp 655

30. What to propagate? • Notifications • Updated data • Update operations Distributed Systems - Comp 655

31. Who initiates propagation? • Server (push-based protocol) • Client (pull-based protocol) Distributed Systems - Comp 655