Local Heap Semantics and its Applications

1. Local Heap Semanticsand its Applications Noam Rinetzky Tel Aviv University Joint work with Jörg Bauer Universität des Saarlandes Thomas Reps University of Wisconsin Mooly Sagiv Tel Aviv University Reinhard Wilhelm Universität des Saarlandes Eran Yahav IBM Watson

2. Motivation • Verify heap intensive programs • Imperative programs with procedures • Recursive data structures • Lists • Trees • …

3. class List { List n; } main() { List x=null, y=null; int k = getLen(); x = create(k); y = reverse(x); } … reverse: reverses terminates No null dereferences No memory leaks k=4 x y x and y point to the same list k=4 n n n x y y points to an acyclic list k=4 n n n x y Motivation arbitrary k

4. Checking heap properties is undecidable What is the problem? • Recursive procedures • Unbounded number of activation records • Dynamic allocation • Unbounded number of objects

5. Our approach • Use abstractions • Over-approximation algorithms • Effective (termination) • Every verified property holds (sound) • May not prove all properties (incomplete)

6. call p(x); X y g t Main idea • Procedures as heap transformers X y g t

7. Main idea • Procedures as local heap transformers

8. x x X X y g t Main idea • Procedures as local heap transformers call p(x); y g t

9. Operational semantics   Abstract transformer Abstract Interpretation[Cousot and Cousot]

10. Operational semantics Abstract transformer ’ ’ Introducing local heap semantics ~ Local heap Operational semantics

11. Arbitrary programs Complicated Restriction on aliasing Simple Main Results POPL’05 SAS’05 • Non standard concrete operational semantics • Sequential programs • Local heap • Storeless • Good for heap abstractions • Observational equivalent with “standard” global store-based heap semantics • E.g., Java • Abstractions • Shape Analysis: singly-linked lists • May-alias [Deutsch, PLDI 04] • Abstractions • Shape Analysis: singly-linked lists an trees • Sorting: quickSort

12. Outline • Motivation • Crash course in shape analysis • Local heap semantics • Local heap abstractions

13. n n n t t t empty x x x n t t t x x n n n t t t x x n n n t t t x x x return x class List { List n; } Collecting semantics x = null ? F T t =new List(); t.n=x; x = t

14. n n x Canonical abstraction n n n x

15. n n t t t empty x x n x n n t t t t n x x x n n n n t t t t n x x x n n n t t t t x n x x n x return x class List { List n; } Shape analysis in action x = null ? F T t =new List(); t.n=x; x = t

16. Outline • Motivation • Crash course in shape analysis • Local heap semantics • Local heap abstractions

17. Programming Model • Single threaded • Procedures • Value parameters • Recursion • No explicit addressing (&, cast) • Heap • Recursive data structures • Destructive update

18. p p x p p x y g t Local heaps call f(x) y g t

19. p p p p x y g t Cutpoints ? x call f(x) y g t

20. Cutpoints • Objects that separate the part of the heap a procedure can access from the rest of the heap • Excluding objects pointed to by a parameter n n n x q n n y n n g z=f(x)

21. Memory state: Val = Addresses  Atoms Env: Var Val Heap: FieldIdAddressVal Natural Easy to identify cutpoint objects Addresses do not affect shape 0x10 n n 0x12 0x12 0x11 0x12 n 0x14 0x0 0x13 0x14 n 0x0 0x10 0x15 … … x0x10 x0x14 n n x Store-based semantics ~

22. n n x.n.n x x.n x y.n.n x n n x y x.n y.n x.n.n y y.n.n y y.n n n y Storeless semantics [Jonkers’81] • No addresses • Memory state: • Object: 2Access paths • Heap: 2Object • Alias Analysis y=x x=null

23. Storeless semantics [Jonkers’81] 0x07 n n x.n.n x x.n • No addresses • Memory state: • Object: 2Access paths • Heap: 2Object • Alias Analysis x y=x 0x07 y.n.n x n n x y x.n y.n x.n.n y x=null 0x07 y.n.n y y.n n n y

24. Cutpoint labels • Relate pre-state with post-state • Mark cutpoints at and throughout an invocation

25. reverse p n n n Cutpoint labels • Cutpoint label: the set of access paths that point to a cutpoint • when the invoked procedure starts {p.n} {p} {p.n.n, p.n.n} {p.n.n.n, p.n.n.n} n n n x n n main y n n g z=f(x)

26. reverse p n n n Sharing patterns • Cutpoint labels encode sharing patterns {p.n} {p} {p.n.n, p.n.n} {p.n.n.n, p.n.n.n} n n n n n n x x n n y y n n n g g main main

27. {p}, {p.n}, , reverse p.n.n p n n n {p.n.n, p.n.n}, {p.n.n.n, p.n.n.n} Memory states L = CPL,A

28. Formal semantics Ordinary statements

29. Procedure call semantics

30. p p p x y g t Local-heap storeless semantics {p.n.n, p.n.n} {p.n.n.n, p.n.n} {x.n.n.n, y.n.n} x call f(x) y g t

31. Observational Equivalence • Programs cannot distinguish between global heap store-based semantics and local-heap storeless semantics • Same executions • Same observed equalities

32. Observational Equivalence • A local store-less state L and a global store-based G are observationally equivalent when for every access paths ,  •  =  L(L)   =   G(G)

33. Main theorem: semantic equivalence • L  L (Local-heap Storeless Semantics) • G  G (Global-heap Store-based Semantics) • L and G observationally equivalent st,L  ’Lst,G  ’G LSL GSB ’L and ’G areobservationally equivalent

34. Corollaries • Preservation of invariants • Assertions:  =  • Detection of memory leaks

35. Outline • Motivation • Crash course in shape analysis • Local heap semantics • Local heap abstractions

36. Applications • Justify soundness of static analysis • May-alias analysis • Shape Analysis • Compile-time garbage collection

37. Shape abstraction • Abstract memory states represent unbounded concrete memory states • Conservatively • In a bounded way

38. z y n x n n n n t Canonical abstraction y z n n n n n x n n t

39. z y n x n n n n t Canonical abstraction y z n n n n n x n n n t

40. z y n n n n n rx rx,ry rz rz rx x n n rt rt t Abstract memory states (with reachability) z y n n n n n rz rx rx rx rx,ry rz rz rz rx rx rx rx,ry rz rz x n n rt rt rt rt rt rt t

41. z y n x n n n n t The importance of reachability:Call append(y,z) z y n n n n n rz rx rx rx rx,ry rz rz x n n rt rt rt t y z n n n n n x rx rx,ry rz rz rx n n rt rt t

42. Unbounded state Objects Cutpoint labels Canonical abstraction Summarizes objects Summarizes labels n n n n p Cutpoints and abstraction y1 y2 n n n n call f(x) x

43. p p x p p x y g t Cutpoint abstraction: ramifications call f(x) y g t

44. x x x x x ? y g t Cutpoint freedom POPL ’05 SAS ’05 call p(x); y g t

45. Interprocedural shape analysis p p x x y Tabulation exits POPL’05: marking cutpoint SAS’05: verifying cutpoint freedom call f(x) y

46. Interprocedural shape analysis p p p x p x y Analyze f No tabulation call f(x) y

47. Prototype implementation

48. Iterative vs. Recursive (SLL) 585

49. Inline vs. Procedural abstraction // Allocates a list of // length 3 List create3(){ … } main() { List x1 = create3(); List x2 = create3(); List x3 = create3(); List x4 = create3(); … }

50. Related Work • Interprocedural shape analysis • Rinetzky and Sagiv, CC ’01 • Chong and Rugina, SAS ’03 • Jeannet et al., SAS ’04 • Hackett and Rugina, POPL ’05 • Local Reasoning • Ishtiaq and O’Hearn, POPL ‘01 • Reynolds, LICS ’02 • Encapsulation • Noble et al. IWACO ’03 • ...