Sequential reductions for verifying serializability

1. Sequential reductions for verifying serializability Hagit Attiya Technion & EPFL G. Ramalingam MSR India Noam Rinetzky University of London

2. The goal Verify concurrent data structures • Pre-execution static analysis E.g., linked list with hand-over-hand locking • memory safety (no memory leaks) • shape invariants (it’s a list) • “atomicity” (serializability) Find sequential reductions • Consider only sequential executions • But conclude that properties hold in all executions

3. Back-of-envelop estimate of gain Static analysis of a linked-list algorithm [Amit, Rinetzky, Reps, Sagiv, Yahav, CAV 2007] • Verifies e.g., memory safety, sortedness, pointed-to by a variable, heap sharing

4. ~ ~ ~ ~ ~ ~ ~ ~ ~ Serializability [Papadimitriou ‘79] interleaved execution operation ~ thread-local views complete non-interleaved execution

5. Serializability assists verification Concurrent code M Π = all executions of M cni-Π = all complete non-interleaved executions of M  Π φ = some thread-local property If M is serializable Then Π ⊨ φ cni-Π ⊨ φ

6. How do we know that M is serializable, w/o considering all executions? I.e., by checking only cni-executions If M is serializable Then Π ⊨ φ cni-Π ⊨ φ

7. Special (and common) case: Disciplined programming with locks Guard access to data with locks Access data only after getting a lock for it (well-lockedness) Follow a locking protocol that guarantees conflict serializability E.g., two-phase locking (2PL) or tree locking (TL)

8. H Two-phase locking [Papadimitriou `79] • Locks acquire (grow) phase followed by locks release (shrink) phase • No lock is acquired after some lock is released t2 t1 t1 t1 t1

9. H Tree (hand-over-hand) locking [Bayer & Scholnick ‘77] [Kedem & Sliberschatz ‘76] [Smadi ‘76] • Except for the first lock, acquire a lock only when holding the lock on its parent • No lock is acquired after being released t2 t1 t1 t1

10. H Tree (hand-over-hand) locking [Bayer & Scholnick ‘77] [Kedem & Sliberschatz ‘76] [Smadi ‘76] • Except for the first lock, acquire a lock only when holding the lock on its parent • No lock is acquired after being released t2 t1 t1 t2

11. void p() { acquire(B) B = 0 int b = B release(B) if (b) Y = 2 acquire(A) } void q() { acquire(B) B = 1 release(B) } Not two-phase locked But only in interleaved executions Yes! • for databases • concurrency control monitor ensures that M follows the locking policy at run-time  M is serializable No! • for static analysis • no central monitor How to (statically) verify that M follows a locking policy?

12. Our approach Applies for local conflict-serializable (LCS) locking protocols • Use only thread-local information (see next) E.g., two phase locking, tree locking, (dynamic) DAG locking… But not protocols that rely on a concurrency control monitor!

13. Thread-local properties A thread-owned view contains the values of thread’s local variables & global variables locked by it A property φ is thread-localif it • Can be expressed in terms of thread-owned views • Is prefix closed A thread-local property of an execution holds in every execution indistinguishable from it

14. non-interleaved execution Our contribution: Easy step cni-Π = all complete non-interleaved executions of M  Π Two phase locking Tree locking Dynamic tree locking Dynamic DAG locking For any LCS locking policy LP Π ⊨ LP ni-Π ⊨ LP For any thread-local property φ Π ⊨ φ ni-Π ⊨ φ

15. Ni-reduction: Proof sketch If all ni-executions follow LP  all executions follow LP σ is the shortest execution that does not follow LP • σ’ follows LP, guarantees conflict-serializability • there is a ni-execution that is “equivalent” to σ’ (t,e) σ σ’

16. σ’ni (t,e) σ’ (t,e) Ni-reduction: Proof sketch (cont.) • there is a ni-execution that is “equivalent” to σ’ • there is a ni-execution that is “equivalent” to σwhere LP is violated σni

17. almost complete non-interleaved execution Further reduction acni-Π = all almost-complete non-interleaved executions of M  Π For any LCS locking policy LP Π ⊨ LP acni-Π ⊨ LP

18. Acni-reduction: A complication Need to argue about termination Observe Y == 1 & violates 2PL Y is set to 1 & the method enters an infinite loop int X=0, Y=0 void p() { acquire(Y) y = Y release(Y); if (y ≠ 0) acquire(X) X = 3 release(X) } void q() { if (random(5) == 3){ acquire(Y) Y = 1 release(Y) while (true) nop } }

19. Acni-reduction: Termination  Can use sequential reduction to verify termination For any “terminating” LCS locking policy LP Π ⊨ LP acni-Π ⊨ LP

20. Acni-reduction: Proof ideas Start from a ni-execution (and rely on the previous, ni-reduction to get there) Create its equivalent completion, if possible Not always possible, e.g., Does not access variables accessed by later threads v t1:lock(v), t1:lock(u), t2:lock(u) u

21. Implications for statis analysis • Pessimistic analysis (over approximate) • Analyze a module from every possible state • Semi-optimistic analysis • Analyze a module only from states that occur after a sequence of modules ran one after the other • Optimistic analysis (precise) • Analyze a module only from states that occur after a sequence of modules ran to completion (one after the other) Ni-reduction Acni-reduction

22. Initial analysis results Shape analysis of hand-over-hand lists * Does not verify sortedness of list and fails to verify linearizability in some cases Shape analysis of hand-over-hand trees(for the first time)

23. What’s next? • Further extensions, esp., for software transactional memory • aborted transactions • non-locking non-conflict based serializability (e.g., using timestamps) • Combine with other reductions [Guerraoui, Henzinger, Jobstmann, Singh]

24. And beyond… • The cost of verifying adherence to a locking policy in an acni- / ni-execution • Automatic insertion of lock acquire / release commands … Or fences?

25. Thank you!