Model-Checking JML Specifications with Bogor

1. Model-Checking JML Specifications with Bogor SAnToS Laboratory, Kansas State University, USA CASSIS 2004, Marseille, France http://bogor.projects.cis.ksu.edu Robby Edwin Rodríguez http://spex.projects.cis.ksu.edu http://jmleclipse.projects.cis.ksu.edu Matthew B. Dwyer John Hatcliff http://bandera.projects.cis.ksu.edu Support Boeing Honeywell Technology Center IBM Intel Lockheed Martin NASA Langley Rockwell-Collins ATC Sun Microsystems US Army Research Office (ARO) US National Science Foundation (NSF) US Department of Defense Advanced Research Projects Agency (DARPA)

2. Bandera:An Open Toolset for Model Checking Concurrent Java Programs Eclipse Platform Optimization Control ? void add(Object o) { buffer[head] = o; head = (head+1)%size; } Object take() { … tail=(tail+1)%size; return buffer[tail]; } Transformation & Abstraction Tools Java Source Error Trace Mapping Bandera Checker Input Slicing Specification Abstract Interpretation Bogor Static Analysis Checker Output

3. SpEx Project — Goals • specification languages should • have a rich set of primitives for observing program state • heap-allocated objects, concurrency, etc. • make it easy to write useful specifications • support lightweight and deep-semantic specifications • be checkable using a variety of analysis techniques • model checking, testing, etc. • We are investigating several languages • JML (current focus), OCL, AAL, etc.

4. JML Reasoning Tools and Technologies other tools such as JACK,…

5. JML Reasoning Tools and Technologies Environment Model Checking/Testing Theorem Proving … m(…) { } … m(…) { assume pre-conditions … … … … … … prove post-conditions } checking that specifications are satisfied for particular traces generated by the environment (test harness) manipulate formulas

6. What is it? Why is it useful? What about its existing algorithms suggests that it might be good for checking JML? Bogor Questions…

7. Bogor (Buitenzorg)

8. Bogor – Software Model Checking Framework

9. unbounded dynamic creation of threads and objects automatic memory management (garbage collection) virtual methods, … …, exceptions, etc. supports virtually all of Java thread & heap symmetry compact state representation partial order reduction techniques driven by object escape analysis locking disciplines Bogor – Direct support for OO software Extensive support for checking concurrent OO software Software targeted algorithms… Direct support for…

10. Bogor – Eclipse-based Tool Components Next generation of Bandera Java Model-checking Tool Set Cadena CORBA Component Model verification SpEx JML Verification, etc. Tool Development Framework Architecture allows encapsulation/integration with other verification tools using IBM’s Eclipse Integrated Development Environment

11. Domain-Specific Model-Checking —Bogor Customized To Cadena Bogor -- Extensible Modeling Language Sets Threads, Objects, Methods, Exceptions, etc. RT CORBA Event Service API Abstraction + + Queues Tables Core Modeling Language Extensions Domain-specific Abstractions Bogor -- Customizable Checking Engine Modules …existing modules… Scheduling Strategy State-space Exploration State-space Representation Core Checker Modules Customized Checker Modules Real-time Scheduling Quasi-cyclic Search Partial State Representation

12. Bogor Customized To Bandera Bogor – Feature-rich Modeling Language Threads, Objects, Methods, Exceptions, etc. Core Modeling Language Bogor -- Customizable Checking Engine Modules …existing modules… Scheduling Strategy State-space Exploration State Representation Core Checker Modules Customized Checker Modules Partial Order Reduction Depth-first Search Symm. Reduc. &Compression

13. Bogor’s Heap Representation Heap …sort walks over heap, canonicalizes, and collects info Canonical heap Key Points… State …explicit heap representation …after each transition, a topological sort gives heap objects a canonical order …transition may create new objects, garbage, etc. …garbage is eliminated …precise heap model …precise alias information …have access to all visited states (but, efficiently stored using collapse compression)

14. Bogor’s Heap Representation — Enables JML Specs Check Heap …sort walks over heap, canonicalizes, and collects info Canonical heap Key Points… State … many JML features are easy to support in Bogor …precise heap model (c.f., \reach) …transition may create new objects, garbage, etc. …precise alias information (c.f., \modifies) …can easily compare objects in methods pre/post-states (c.f., \old)

15. JML Language Coverage large language coverage…

16. Doug Lea’s LinkedQueue Example publicclassLinkedNode { publicObjectvalue; publicLinkedNodenext; publicLinkedNode(Objectx) { value=x; } } publicclass LinkedQueue { protectedfinal Object putLock; protected LinkedNode head; protected LinkedNode last = head; protectedint waitingForTake = 0; public LinkedQueue() { putLock = new Object(); head = new LinkedNode(null); } publicboolean isEmpty() { synchronized (head) { return head.next == null; } } publicvoidput(Objectx){ if(x==null) thrownewIllegalArgumentException(); insert(x); } protectedsynchronized Object extract() { synchronized (head) { Object x = null; LinkedNode first = head.next; if (first != null) { x = first.value; first.value = null; head = first; } return x; } } protectedvoid insert(Object x) { synchronized (putLock) { LinkedNode p = new LinkedNode(x); synchronized (last) { last.next = p; last = p; } if (waitingForTake > 0) putLock.notify(); return; } } public Object take() { Object x = extract(); if (x != null) return x; else … } allows a high degree of concurrency

17. Doug Lea’s LinkedQueue Example head LQ LN last putLock A state with two threads and a LinkedQueue object

18. Doug Lea’s LinkedQueue Example head LQ LN last putLock The red thread creates a new object

19. Doug Lea’s LinkedQueue Example put() head LQ LN last putLock …and invokes put(), which invokes insert()

20. Doug Lea’s LinkedQueue Example put() head LQ LN last putLock insert() acquires the lock on putLock

21. Doug Lea’s LinkedQueue Example p put() value head LQ LN LN last putLock …and creates a new LinkedNode

22. Doug Lea’s LinkedQueue Example p put() value head LQ LN LN last putLock …then it locks the LinkedNode pointed by last

23. Doug Lea’s LinkedQueue Example p put() value head next LQ LN LN last putLock …and assigns new LinkedNodeto last.next

24. Doug Lea’s LinkedQueue Example p put() value head next LQ LN LN putLock last last is moved to point to the new LinkedNode

25. Doug Lea’s LinkedQueue Example p put() value head next LQ LN LN putLock last the lock on head’s LinkedNode is released

26. Doug Lea’s LinkedQueue Example put() value head next LQ LN LN putLock last returning from insert(), the local p is now out of scope

27. Doug Lea’s LinkedQueue Example put() value head next LQ LN LN putLock last and the lock on putLock’s object is released

28. Doug Lea’s LinkedQueue Example value head next LQ LN LN putLock last The red thread finishes executing the put() method

29. Doug Lea’s LinkedQueue Example value head next LQ LN LN putLock last and it removes the reference to the new object, done!

30. Doug Lea’s LinkedQueue Example value value head next next LQ LN LN LN putLock last another object is added

31. Doug Lea’s LinkedQueue Example value value head next next LQ LN LN LN take() putLock last The blue thread invokes take(), which invokes extract()

32. Doug Lea’s LinkedQueue Example value value head next next LQ LN LN LN take() putLock last the LinkedQueue is locked

33. Doug Lea’s LinkedQueue Example value value head next next LQ LN LN LN take() putLock last the head’s LinkedNode is also locked

34. Doug Lea’s LinkedQueue Example value value head next next LQ LN LN LN take() putLock last first head.next is assigned to the local first

35. Doug Lea’s LinkedQueue Example x value value head next next LQ LN LN LN take() putLock last first first.value is assignedto the local x

36. Doug Lea’s LinkedQueue Example x value head next next LQ LN LN LN take() putLock last first first.value is assigned null

37. Doug Lea’s LinkedQueue Example x head value next next LQ LN LN LN take() putLock last first head is moved to the next LinkedNode

38. Doug Lea’s LinkedQueue Example x head value next LQ LN LN take() putLock last the LinkedNode is GC’ed (after its lock released)

39. Doug Lea’s LinkedQueue Example value head next LQ LN LN putLock last x the state after take() is finished

40. Assessments — LinkedQueue • put()and take() • can be done concurrently • if the size of the LinkedQueue is greater than 0 • they use different locks to protect object accesses • put() synchronizes on putLock and last • take() synchronizes on the LinkedQueue object and head • are mutually exclusive • if the size is 0 • synchronize on the same LinkedNode • head == last • reasoning about them becomes very complex

41. JML & Concurrency Issues • Pre-/post-conditions • check points in a concurrent setting • Functional and synchronization aspects • difficulty when specifying methods • Model checking post-conditions with \old()

42. LinkedQueue Example (JML) /*@behavior @requiresx!=null; @ensurestrue; @alsobehavior @requiresx==null; @signals(Exceptione)einstanceofIllegalArgumentException; @*/ publicvoidput(Objectx){ if(x==null) thrownewIllegalArgumentException(); insert(x); } protectedsynchronizedObjectextract(){ synchronized(head){ returnrefactoredExtract(); } } /*@behavior @assignablehead,head.next.value; @ensures\result==null||(\existsLinkedNoden; @\old(\reach(head)).has(n); @n.value==\result @&&!(\reach(head).has(n))); @*/ protectedObjectrefactoredExtract(){ Objectx=null; LinkedNodefirst=head.next; if(first!=null){ x=first.value; first.value=null; head=first; } returnx; } /*@behavior @requiresx!=null; @ensureslast.value==x&&\fresh(last); @*/ protectedvoidinsert(Objectx){ synchronized(putLock){ LinkedNodep=newLinkedNode(x); synchronized(last)refactoredInsert(p); if(waitingForTake>0)putLock.notify(); return; } } } publicclass LinkedNode { public Object value; public LinkedNode next; /*@behavior @ensuresvalue==x; @*/ public LinkedNode(Object x) { value = x; } } publicclass LinkedQueue { protectedfinal/*@non_null@*/ Object putLock; protected/*@non_null@*/ LinkedNode head; protected/*@non_null@*/ LinkedNode last = head; protectedint waitingForTake = 0; //@instanceinvariantwaitingForTake>=0; //@instanceinvariant\reach(head).has(last); /*@behavior @assignablehead,last,putLock,waitingForTake; @ensures\fresh(head,putLock)&&head.next==null; @*/ public LinkedQueue() { putLock = new Object(); head = new LinkedNode(null); } /*@behavior @ensures\result<==>head.next==null; @*/ publicboolean isEmpty() { synchronized (head) { return head.next == null; } } /*@behavior @requiresn!=null; @assignablelast,last.next; @*/ protectedvoid refactoredInsert(LinkedNode n) { last.next = n; last = n; }

43. LinkedQueue Example (JML) /*@behavior @requiresx!=null; @ensurestrue; @alsobehavior @requiresx==null; @signals(Exceptione)einstanceofIllegalArgumentException; @*/ publicvoidput(Objectx){ if(x==null) thrownewIllegalArgumentException(); insert(x); } protectedsynchronizedObjectextract(){ synchronized(head){ returnrefactoredExtract(); } } /*@behavior @assignablehead,head.next.value; @ensures\result==null||(\existsLinkedNoden; @\old(\reach(head)).has(n); @n.value==\result @&&!(\reach(head).has(n))); @*/ protectedObjectrefactoredExtract(){ Objectx=null; LinkedNodefirst=head.next; if(first!=null){ x=first.value; first.value=null; head=first; } returnx; } /*@behavior @requiresx!=null; @ensureslast.value==x&&\fresh(last); @*/ protectedvoidinsert(Objectx){ synchronized(putLock){ LinkedNodep=newLinkedNode(x); synchronized(last)refactoredInsert(p); if(waitingForTake>0)putLock.notify(); return; } } } publicclass LinkedNode { public Object value; public LinkedNode next; /*@behavior @ensuresvalue==x; @*/ public LinkedNode(Object x) { value = x; } } publicclass LinkedQueue { protectedfinal/*@non_null@*/ Object putLock; protected/*@non_null@*/ LinkedNode head; protected/*@non_null@*/ LinkedNode last = head; protectedint waitingForTake = 0; //@instanceinvariantwaitingForTake>=0; //@instanceinvariant\reach(head).has(last); /*@behavior @assignablehead,last,putLock,waitingForTake; @ensures\fresh(head,putLock)&&head.next==null; @*/ public LinkedQueue() { putLock = new Object(); head = new LinkedNode(null); } /*@behavior @ensures\result<==>head.next==null; @*/ publicboolean isEmpty() { synchronized (head) { return head.next == null; } } /*@behavior @requiresn!=null; @assignablelast,last.next; @*/ protectedvoid refactoredInsert(LinkedNode n) { last.next = n; last = n; } publicclass LinkedQueue { protectedfinal/*@non_null@*/ Object putLock; protected/*@non_null@*/ LinkedNode head; protected/*@non_null@*/ LinkedNode last = head; protectedint waitingForTake = 0; //@instanceinvariantwaitingForTake>=0; //@instanceinvariant\reach(head).has(last); …

44. LinkedQueue Example (JML) /*@behavior @requiresx!=null; @ensurestrue; @alsobehavior @requiresx==null; @signals(Exceptione)einstanceofIllegalArgumentException; @*/ publicvoidput(Objectx){ if(x==null) thrownewIllegalArgumentException(); insert(x); } protectedsynchronizedObjectextract(){ synchronized(head){ returnrefactoredExtract(); } } /*@behavior @assignablehead,head.next.value; @ensures\result==null||(\existsLinkedNoden; @\old(\reach(head)).has(n); @n.value==\result @&&!(\reach(head).has(n))); @*/ protectedObjectrefactoredExtract(){ Objectx=null; LinkedNodefirst=head.next; if(first!=null){ x=first.value; first.value=null; head=first; } returnx; } /*@behavior @requiresx!=null; @ensureslast.value==x&&\fresh(last); @*/ protectedvoidinsert(Objectx){ synchronized(putLock){ LinkedNodep=newLinkedNode(x); synchronized(last)refactoredInsert(p); if(waitingForTake>0)putLock.notify(); return; } } } publicclass LinkedNode { public Object value; public LinkedNode next; /*@behavior @ensuresvalue==x; @*/ public LinkedNode(Object x) { value = x; } } publicclass LinkedQueue { protectedfinal/*@non_null@*/ Object putLock; protected/*@non_null@*/ LinkedNode head; protected/*@non_null@*/ LinkedNode last = head; protectedint waitingForTake = 0; //@instanceinvariantwaitingForTake>=0; //@instanceinvariant\reach(head).has(last); /*@behavior @assignablehead,last,putLock,waitingForTake; @ensures\fresh(head,putLock)&&head.next==null; @*/ public LinkedQueue() { putLock = new Object(); head = new LinkedNode(null); } /*@behavior @ensures\result<==>head.next==null; @*/ publicboolean isEmpty() { synchronized (head) { return head.next == null; } } /*@behavior @requiresn!=null; @assignablelast,last.next; @*/ protectedvoid refactoredInsert(LinkedNode n) { last.next = n; last = n; } /*@behavior @assignablehead,last,putLock,waitingForTake; @ensures\fresh(head,putLock)&&head.next==null; @*/ public LinkedQueue() { putLock = new Object(); head = new LinkedNode(null); }

45. LinkedQueue Example (JML) /*@behavior @requiresx!=null; @ensurestrue; @alsobehavior @requiresx==null; @signals(Exceptione)einstanceofIllegalArgumentException; @*/ publicvoidput(Objectx){ if(x==null) thrownewIllegalArgumentException(); insert(x); } protectedsynchronizedObjectextract(){ synchronized(head){ returnrefactoredExtract(); } } /*@behavior @assignablehead,head.next.value; @ensures\result==null||(\existsLinkedNoden; @\old(\reach(head)).has(n); @n.value==\result @&&!(\reach(head).has(n))); @*/ protectedObjectrefactoredExtract(){ Objectx=null; LinkedNodefirst=head.next; if(first!=null){ x=first.value; first.value=null; head=first; } returnx; } /*@behavior @requiresx!=null; @ensureslast.value==x&&\fresh(last); @*/ protectedvoidinsert(Objectx){ synchronized(putLock){ LinkedNodep=newLinkedNode(x); synchronized(last)refactoredInsert(p); if(waitingForTake>0)putLock.notify(); return; } } } publicclass LinkedNode { public Object value; public LinkedNode next; /*@behavior @ensuresvalue==x; @*/ public LinkedNode(Object x) { value = x; } } publicclass LinkedQueue { protectedfinal/*@non_null@*/ Object putLock; protected/*@non_null@*/ LinkedNode head; protected/*@non_null@*/ LinkedNode last = head; protectedint waitingForTake = 0; //@instanceinvariantwaitingForTake>=0; //@instanceinvariant\reach(head).has(last); /*@behavior @assignablehead,last,putLock,waitingForTake; @ensures\fresh(head,putLock)&&head.next==null; @*/ public LinkedQueue() { putLock = new Object(); head = new LinkedNode(null); } /*@behavior @ensures\result<==>head.next==null; @*/ publicboolean isEmpty() { synchronized (head) { return head.next == null; } } /*@behavior @requiresn!=null; @assignablelast,last.next; @*/ protectedvoid refactoredInsert(LinkedNode n) { last.next = n; last = n; } /*@behavior @ensures\result<==>head.next==null; @*/ publicboolean isEmpty() { synchronized (head) { return head.next == null; } }

46. Pre/Post-Conditions jmlc generates a wrapper method for each annotated method Figure 4.3, “A Runtime Assertion Checker for the Java Modeling Language”, Y. Cheon

47. Pre/Post-Conditions check invariants and method preconditions Figure 4.3, “A Runtime Assertion Checker for the Java Modeling Language”, Y. Cheon

48. Pre/Post-Conditions call original method Figure 4.3, “A Runtime Assertion Checker for the Java Modeling Language”, Y. Cheon

49. Pre/Post-Conditions check post-conditions Figure 4.3, “A Runtime Assertion Checker for the Java Modeling Language”, Y. Cheon

50. Pre/Post-Conditions At this point a thread can interleave and insert an object in the LinkedQueue; so there actually exists an execution race where the post-condition is violated. /*@behavior @ensures\result<==>head.next==null; @*/ publicbooleanisEmpty(){ synchronized(head){ returnhead.next==null; } } publicbooleanisEmpty() { … booleanrac$result; … rac$result=orig$isEmpty(); checkPost$isEmpty$LinkedQueue(rac$result); returnrac$result; … }