Automated Verification of Concurrent Linked Lists with Counters

1. Automated Verification of Concurrent Linked Lists with Counters Tuba Yavuz-Kahveci and Tevfik Bultan Department of Computer Science University of California, Santa Barbara {tuba,bultan}@cs.ucsb.edu http://www.cs.ucsb.edu/~bultan/composite

2. General Problem • Concurrent programming is difficult and error prone • Sequential programming: states of the variables • Concurrent programming: states of the variables and the processes • Linked list manipulation is difficult and error prone • States of the heap: possibly infinite • We would like to guarantee properties of a concurrent linked list implementation

3. More Specific Problem • There has been work on verification of concurrent systems with integer variables (and linear constraints) • [Bultan, Gerber and Pugh, TOPLAS 99] • [Delzanno and Podelski STTT01] • Use widening based on earlier work of [Cousot and Halbwachs POPL 77] on analyzing programs with integer variables • There has been work on verification of (concurrent) linked lists • [Yahav POPL’01] • What can we do for concurrent systems: • where both integer and heap variables influence the control flow, • or the properties we wish to verify involve both integer and heap variables?

4. Our Approach • Use symbolic verification techniques • Use polyhedra to represent the states of the integer variables • Use BDDs to represent the states of the boolean and enumerated variables • Use shape graphs to represent the states of the heap • Use composite representation to combine them • Use forward-fixpoint computations to compute reachable states • Truncated fixpoint computations can be used to detect errors • Over-approximation techniques can be used to prove properties • Polyhedra widening • Summarization in shape graphs

5. Action Language Verifier Action Language Tool Set Action Language Specification of the Concurrency Component Action Language Parser Composite Symbolic Library Code Generator Omega Library CUDD Package MONA Verified code (Java monitor classes)

6. Outline • Specification of concurrent linked lists • Action Language • Symbolic verification • Composite representation • Approximation techniques • Summarization • Widening • Counting abstraction • Experimental results • Related Work • Conclusions

7. Action Language [Bultan ICSE00] [Yavuz-Kahveci, Bultan ASE01] • A state based language • Actions correspond to state changes • States correspond to valuations of variables • Integer (possibly unbounded), heap, boolean and enumerated variables • Parameterized constants are allowed • Transition relation is defined using actions • Atomic actions: Predicates on current and next state variables • Action composition: synchronous (&) or asynchronous (|) • Modular • Modules can have submodules • Properties to be verified • Invariant(p) : p always holds

8. Composite Formulas: State Formulas • We use state formulas to express the properties we need to check • No primed variables in state formulas • State formulas are boolean combination (, ,,,) of integer, boolean and heap formulas numItems>2 => top.next!=null integer formula heap formula

9. State formulas • Boolean formulas • Boolean variables and constants (true, false) • Relational operators: =,  • Boolean connectives (, ,,,) • Integer formulas (linear arithmetic) • Integer variables and constants • Arithmetic operators: +,-, and * with a constant • Relational operators: =, , > , <, ,  • Boolean connectives (, ,,,) • Heap formulas • Heap variable, heap-variable.selector, heap constant null • Relational operators: =,  • Boolean connectives (, ,,,)

10. Composite Formulas: Transition Formulas • We use transition formulas to express the actions • In transition formulas primed-variables denote the next-state values, unprimed-variables denote the current-sate values pc=checknull and numItems=0 and top’=add and add’.next=null and numItems’=1 and pc’=create and mutex’; current state variables next state variables

11. Transition Formulas • Transition formulas are in the form: boolean-formula  integer-formula heap-transition-formula • Heap transition formulas are in the form: guard-formula  update-formula • A guard formula is a conjunction of terms in the form: id1 = id2 id1 id2 id1.f = id2 id1.f  id2 id1.f = id2.f id1.f  id2.f id1 = null id1  null id1.f = null id1.f  null • An update formula is a conjunction of terms in the form: id’1 = id2 id’1 = id2.f id’1.f = id2 id’1.f = id2.f id’1 = null id’1.f = null id’1= new id’1.f = new

12. module main() heap {next} top, add, get, newTop; boolean mutex; integer numItems; initial: top=null and mutex and numItems=0; module push() enumerated pc {create, checknull,updateTop}; initial: pc=create and add=null; push1: pc=create and mutex and !mutex’ and add’=new and pc’=checknull; push2: pc=checknull and top=null and top’=add and add’.next=null and numItems'=1 and pc’=create and mutex’; push3: pc=checknull and top!=null and add’.next=top and pc’=updateTop; push4: pc=updateTop and top’=add and numItems’=numItems+1 and mutex’ and pc’=create; push: push1 | push2 | push3 | push4; endmodule Variable declarations define the state space of the system Stack Example Predicates defining the initial states Atomic actions: primed variables denote next sate variables Transition relation of the push module is defined as asynchronous composition of its atomic actions

13. Stack (Cont’d) module pop() enumerated pc {copyTopNext, getTop, updateTop}; initial: pc=copyTopNext and get=null and newTop=null; pop1: pc=copyTopNext and mutex and top!=null and newTop’=top.next and !mutex’ and pc’=getTop; pop2: pc=getTop and get’=top and pc’=updateTop; pop3: pc=updateTop and top’=newTop and mutex’ and numItems’=numItems-1 and pc’=copyTopNext; pop: pop1 | pop2 | pop3; endmodule main: pop() | pop() | push() | push(); spec: invariant([numItems=0 => top=null]) spec: invariant([numItems>2 => top->next!=null]) endmodule Transition relation of main defined as asynchronous composition of two pop and two push processes Invariants to be verified

14. Stack (with integer guards) module main() heap {next} top, add, get, newTop; boolean mutex; integer numItems; initial: top=null and mutex and numItems=0; module push() enumerated pc {create, checknull,updateTop}; initial: pc=create and add=null; push1: pc=create and mutex and !mutex’ and add’=new and pc’=checknull; push2: pc=checknull and numItems=0 and top’=add and add’.next=null and numItems’=1 and pc’=create and mutex’; push3: pc=checknull and numItems>0 and add’.next=top and pc’=updateTop; push4: pc=updateTop and top’=add and numItems’=numItems+1 and mutex’ and pc’=create; push: push1 | push2 | push3 | push4; endmodule

15. Outline • Specification of concurrent linked lists • Action Language • Symbolic verification • Composite representation • Approximation techniques • Summarization • Widening • Counting abstraction • Experimental results • Related Work • Conclusions

16. Symbolic Verification: Forward Fixpoint • Forward fixpoint for the reachable states can be computed by iteratively manipulating symbolic representations • We need forward-image (post-condition), union, and equivalence check computations ReachableStates(I: Set of initial states, T: Transition relation) { RS := I; repeat { RSold := RS; RS := RSold forwardImage(RSold, T); } until (RSold = RS) }

17. add top Symbolic Verification: Symbolic Representations • Use a symbolic representation for the sets of states • A boolean logic formula (stored as a BDD) represents the sets of states of the boolean variables: pc=create  mutex • An arithmetic constraint (stored as polyhedra) represents the sets of states of integer variables: numItems>0 • Shape graphs are used to represent the sates of the heap variables and the heap

18. Composite Representation • Each variable type is mapped to a symbolic representation type • Boolean and enumerated types  BDD representation • Integer variables  Polyhedra • Heap variables  Shape graphs • Each conjunct in a transition formula operates on a single symbolic representation • Composite representation: A disjunctive representation to combine different symbolic representations • Union, equivalence check and forward-image computations are performed on this disjunctive representation

19. Composite Representation • A composite representation A is a disjunction where • n is the number of composite atoms in A • tis the number of basic symbolic representations • Each composite atom is a conjunction • Each conjunct corresponds to a different symbolic representation

20. add top Composite Representation: Example A list of shape graphs BDD A list of polyhedra   pc=create  mutex • numItems=2  add top   pc=checkNull  mutex • numItems=2  add top   pc=updateTop  mutex • numItems=2  add top   pc=create  mutex • numItems=3

21. Composite Symbolic Library [Yavuz-Kahveci, Tuncer, Bultan TACAS01], [Yavuz-Kahveci, Bultan STTT02] • Our library implements this approach using an object-oriented design • A common interface is used for each symbolic representation • Easy to extend with new symbolic representations • Enables polymorphic verification • As a BDD library we use Colorado University Decision Diagram Package (CUDD) [Somenzi et al] • As an integer constraint manipulator we use Omega Library [Pugh et al] • For encoding the states of the heap variables and the heap we use shape graphs encoded as BDDs (using CUDD)

22. CompSym –representation: list of comAtom + union() • • • compAtom ShapeGraph –atom: *Symbolic –atom: *Symbolic Composite Symbolic Library: Class Diagram Symbolic +union() +isSatisfiable() +isSubset() +forwardImage() BoolSym HeapSym IntSym –representation: BDD –representation: list of ShapeGraph –representation: list of Polyhedra +union() • • • +union() • • • +union() • • • CUDD Library OMEGA Library

23. Satisfiability Checking for Composite Representation is boolean isSatisfiable(CompSym A) for each compAtoma in Ado ifa is satisfiable then return true return false boolean isSatisfiable(compAtom a) for each symbolic representation tdo ifat is not satisfiable then return false return true Satisfiable? isSatisfiable? or isSatisfiable? is is is is and and  Satisfiable? Satisfiable? Satisfiable? Satisfiable?

24. • • • Forward Image for Composite Representation R: A: CompSymforwardImage(Compsym A, transitionRelation R) CompSym C; for each compAtom a in Ado for each atomic action r in Rdo insert forwardImage( a,r ) into C returnC C:

25. Forward Image for Composite Atom compAtomforwardImage(compAtom a, atomic action r) for each symbolic representation typetdo replace at by forwardImage(at , rt) returna r: a:

26. pc=updateTop and pc’=create and mutex’ numItems’=numItems+1 top’=add add top   pc=create  mutex • numItems=3 Forward-Image Computation: Example add top   pc=updateTop  mutex • numItems=2

27.  add top   pc=checkNull  mutex • numItems=0  add top   pc=create  mutex • numItems=1  add top   pc=checkNull  mutex • numItems=1 Forward–Fixpoint Computation (Repeatedly Applies Forward-Image) add top   pc=create  mutex • numItems=0

28.  add top   pc=updateTop  mutex • numItems=1  add top   pc=create  mutex • numItems=2  add top   pc=checkNull  mutex • numItems=2  add top   pc=updateTop  mutex • numItems=2

29. add top   pc=create  mutex • numItems=3  . . .

30. Forward-Fixpoint does not Converge • We have two reasons for non-termination • integer variables can increase without a bound • the number of nodes in the shape graphs can increase without a bound • The state space is infinite • Even if we ignore the heap variables, reachability is undecidable when we have unbounded integer variables • So, we use conservative approximations

31. Outline • Specification of concurrent linked lists • Action Language • Symbolic verification • Composite representation • Approximation techniques • Summarization • Widening • Counting Abstraction • Experimental results • Related Work • Conclusions

32. p RS + RS “The property is satisfied” Conservative Approximations • To verify or falsify a property p • Compute a lower ( RS  ) or an upper ( RS + ) approximation to the set of reachable states • There are three possibilities:

33. reachable sates which violate the property p RS RS  “The property is false” p RS RS + RS  “I don’t know” Conservative Approximations

34. Computing Upper and Lower Bounds for Reachable States • Truncated fixpoint computation • To compute a lower bound for a least-fixpoint computation • Stops after a fixed number of iterations • Widening • To compute an upper bound for the least-fixpoint computation • We use a generalization of the polyhedra widening operator by [Cousot and Halbwachs POPL’77] • Summarization • Generate heap nodes in the shape graphs which represent more than one concrete node • Materialization: we need to generate concrete nodes from the summary nodes when needed

35. Summarization • The nodes mapped to a summary node form a chain • No heap variable points to any concrete node that is mapped to a summary node • Each concrete node mapped to a summary node is only pointed by one pointer • During summarization, we also introduce an integer variable which counts the number of concrete nodes mapped to a summary node ...

36. After summarization, it becomes: add top   pc=create  mutex • numItems=3  summarycount=2 a new integer variable representing the number of concrete nodes encoded by the summary node summary node Summarization Example add top   pc=create  mutex • numItems=3

37. Summarization • Summarization guarantees that the number of different shape graphs that can be generated are finite • However, the summary-counts can still increase without a bound • We use polyhedral widening operation to force the fixpoint computation to convergence

38.  add top • numItems=3 •  summaryCount=2  pc=checkNull  mutex   add top • numItems=3 •  summaryCount=2   pc=updateTop  mutex  add • numItems=4 •  summaryCount=2  top pc=create  mutex  We need to do summarization Let’s Continue the Forward-fixpoint  add top • numItems=3 •  summaryCount=2  pc=create  mutex 

39. After summarization, it becomes: add • numItems=4 •  summaryCount=3  top pc=create  mutex  Summarization add • numItems=4 •  summaryCount=2  top pc=create  mutex 

40. Simplification • After each fixpoint iteration we try to merge as many composite atoms as possible • For example, following composite atoms can be merged add top • numItems=3 •  summaryCount=2  pc=create  mutex  add • numItems=4 •  summaryCount=3  top pc=create  mutex 

41. = add • (numItems=4 • summaryCount=3 • numItems=3  summarycount=2)  top pc=create  mutex  Simplification add top • numItems=3 •  summaryCount=2  pc=create  mutex   add • numItems=4 •  summaryCount=3  top pc=create  mutex 

42. = add   top pc=create  mutex • numItems=summaryCount+1 • 3  numItems • numItems  4 Simplification on the integer part add • (numItems=4 • summaryCount=3 • numItems=3  summaryCount=2)  top pc=create  mutex 

43. Widening • Forward-fixpoint computation still will not converge since numItems and summaryCount keep increasing without a bound • We use the widening operation: • Given two composite atoms c1 and c2 in consecutive fixpoint iterates, assume that c1 = b1 i1 h1 c2 = b2 i2 h2 where b1 = b2 and h1 = h2 and i1  i2 • Also assume that i1 is a single polyhedron (i.e. a conjunction of arithmetic csontraints) and i2 is also a single polyhedron

44. Widening • Then • i1 i2 is defined as: all the constraints in i1 which are also satisfied by i2 • Replace i2 with i1 i2 in c2 • This gives a majorizing sequence to the forward-fixpoint computation

45. = add   top pc=create  mutex • numItems=summaryCount+1 • 3  numItems Now, the forward-fixpoint converges Widening Example add   top pc=create  mutex • numItems=summaryCount+1 • 3  numItems • numItems  4  add   top pc=create  mutex • numItems=summaryCount+1 • 3  numItems • numItems  5

46. Dealing with Arbitrary Number of Processes • Use counting abstraction [Delzanno CAV’00] • Create an integer variable for each local state of a process • Each variable will count the number of processes in a particular state • Local states of the processes have to be finite • Shared variables of the monitor can be unbounded • Counting abstraction can be automated

47. Stack After Counting Abstraction Variables for counting the number of processes in each state Parameterized constant representing the number of processes module main() heap top, add, get, newTop; boolean mutex; integer numItems; integer CreateC, ChecknullC,UpdateTopC; parameterized integer numProcesses; initial: top=null and mutex and numItems=0 and CreateC=numProcesses and ChecknullC=0 and UpdateTopC=0; restrict: numProcesses>0; module push() //enumerated pc {create, checknull,updateTop}; initial: add=null; push1: CreateC>0 and mutex and !mutex' and add'=new and CreateC'=CreateC-1 and ChecknullC'=ChecknullC+1; push2: ChecknullC>0 and top=null and top'=add and add'->next=null and numItems'=1 and ChecknullC'=ChecknullC-1 and CreateC'=CreateC+1 and mutex'; push3: ChecknullC>0 and top!=null and add'->next=top and ChecknullC'=ChecknullC-1 and UpdateTopC'=UpdateTopC+1; push4: UpdateTopC>0 and top'=add and numItems'=numItems+1 and mutex' and UpdateTopC'=UpdateTopC-1 and CreateC'=CreateC+1; push: push1 | push2 | push3 | push4; endmodule Initialize initial state counter to the number of processes. Initialize other states to 0. When local state changes, decrement current local state counter and increment next local state counter

48. Verified Properties

49. Experimental Results - Verification Times

50. Related Work • There is a lot of work on Shape analysis, I will just mention the ones which directly influenced us: • [Sagiv,Reps, Wilhelm TOPLAS’98], [Dor, Rodeh, Sagiv SAS’00] • Verification of concurrent linked lists with arbitrary number of processes in [Yahav POPL’01] • [Lev-Ami, Reps, Sagiv, Wilhelm ISSTA 00] use 3-valued logic and instrumentation predicates to verify properties that cannot be expressed in our framework, however, our approach does not require instrumentation predicates • Deutch used integer constraint lattices to compute aliasing information using symbolic access paths [Deutch PLDI’94] • Use of BDDs goes back to symbolic model checking [McMillan’93] and verification with arithmetic constraints goes back to [Cousot and Halbwachs’77]