Towards a Compositional SPIN Corina Păsăreanu QSS, NASA Ames Research Center Dimitra Giannakopoulou RIACS/USRA, NASA Ames Research Center
C1 C1 C2 M2 C2 M1 models Cost of detecting/fixing defects increases Integration issues handled early Project Overview Main objective: • An integrated environment that supports software development and verification/validation throughout the lifecycle; detectintegration problems early, prior to coding Approach: • Compositional (“divide and conquer”) verification, for increased scalability, at design level • Use design level artifacts to improve/aid coding, testing and fault containment Compositional Verification Requirements Design Coding Testing Deployment implementations
Towards a Compositional SPIN • Learning based compositional analysis for increased scalability [TACAS’03] – LTSA tool • Contributions • Generic tool architecture for learning based assume guarantee reasoning • Handles multiple components • Uses SPIN to answer model checking questions • Other tools can be used (e.g. Java PathFinder) • Heuristic for automated generation of interface specifications • Application to realistic resource arbiter for a spacecraft • Significant memory gains over “traditional” non-compositional model checking
Outline • Introduction • Background: assume-guarantee reasoning and learning • Tool architecture • Implementation • Using SPIN for answering learning questions • Promela subset • Case study: MER Arbiter model • Description, results, discussion • Conclusions and future work
satisfies P? Compositional Verification Does system made up of M1 and M2 satisfy property P? M1 • Check P on entire system: too many states! • Use the natural decomposition of the system into its components to break-up the verification task • Check components in isolation: Does M1 satisfy P? • Typically a component is designed to satisfy its requirements in specific contexts / environments • Assume-guarantee reasoning: introduces assumption A representing M1’s “context” A M2
satisfies P? • A M1 P • true M2 A • true M1 || M2 P “discharge” the assumption Assume-Guarantee Rules • Reason about triples: A M P The formula is true if whenever M is part of a system that satisfies A, then the system must also guarantee P M1 • Simplest assume-guarantee rule • We can also handle symmetric rules A M2 How do we come up with the assumption? (usually a difficult manual process)
Approaches • Infer assumptions automatically • Two solutions developed • Algorithmic generation of assumption (controller); knowledge of environment is notrequired [ASE’02] • Incremental assumption computation based on counterexamples, learning and knowledge of environment [TACAS’03, SAVCBS’03 (symmetric rules)]
Formalisms • Components modeled as finite state machines (FSM) • FSMs assembled with parallel composition operator “||” • Synchronizes shared actions, interleaves remaining actions • A property P is a FSM • P describes all legal behaviors • Perr– determinize & complete P; bad behaviors lead to “error” • Component M satisfies P iff error state unreachable in (M || Perr) • Assume-guarantee reasoning • Assumptions and guarantees are FSMs • A M P holds iff error state unreachable in (A || M || Perr) • The alphabet of A, A, contains all environment actions that appear in P
Example Input Ordererr in send in ack || out out in Output send out ack
Computing the Weakest Assumption • Given component M, property P, and the interface of M with its environment, generate the weakest environment assumption A such that: assuming A, M╞ P • Weakest means that for all environments E: (E || M╞P) IFFE╞A
A M1 P • true M2 A • true M1 || M2 P Learning for Assume-guarantee Reasoning • Use an off-the-shelf learning algorithm to build appropriate assumption for the rule • Process is iterative • Assumptions are generated by querying the system, and are gradually refined • Queries are answered by model checking • Refinement is based on counterexamples obtained by model checking • Termination is guaranteed • Extended with symmetric rules
Unknown regular language U Learning with L* • L* algorithm by Angluin, improved by Rivest & Schapire • Learns an unknown regular language U (over alphabet) and produces a DFA A such that L(A) = U • Uses a teacher to answer two types of questions true L* query: string s is s in U? false remove string t false • conjecture:Ai true output DFA A such that L(A) = U is L(Ai)=U? false add string t
s M1 P Ai M1 P true M2 Ai true error? Learning Assumptions • A M1 P • true M2 A • true M1 || M2 P • Use L* to generate candidate assumptions • A = (M1P) M2 true Model Checking L* query: string s false remove counterex. false • conjecture:Ai true true P holds in M1 || M2 false yes P violated add counterex. no
Characteristics • Terminates with minimal automaton A for U • Generates DFA candidates Ai: |A1| < | A2| < … < |A| • Produces at most n candidates, where n = |A| • # queries:(kn2 + n logm), • m is size of largest counterexample, k is size of alphabet • For n components (M1 || M2 || … || Mn) apply algorithm recursively: • Ai M1 P as before • true M2 || … || MnAi invokes the framework
true s M1 P query: string s false Learning Interface Specifications • Compute an assumption (as weak as possible) for a component M1 to guarantee some property P, when environment is not available • A = P L* counterexample false • conjecture:Ai Ai M1 P true true return Ai true P Ai counterexample not accurate with respect to !P || !M1
Implementation in SPIN • Stand-alone application • Invokes SPIN for queries and conjectures • Handles only a Promela subset • Components are processes • Communication through rendezvous channels • Safety properties • SPIN trace assertions • Checking assume-guarantee triples • Encode assumptions into environment processes that run in parallel with components
Case Study • Model derived from MER Resource Arbiter • Local management of resource contention between resource consumers (e.g. science instruments, communication systems) • Avoid simultaneous conflicts (e.g. driving while capturing a camera image are mutually incompatible) • Enforce orderly activity in accordance with predefined priorities encoded in a lookup table • 5 resources • Comm, Drive, PanCam, Arm, Rat • 5 User threads • Non-deterministically decide to use any of the 5 resources • ~3000 LOC • Checked several safety properties
U5 ARB Request, Cancel Grant, Deny Rescind U4 U3 U2 U1 Arbiter Architecture Property P • Mutual exclusion between resources • Comm and Drive can not be used at the same time
U1 P A1 A2 A1 U3 A2 A3 A4 A3 U2 U4 U5 A5 A4 ARB A5 Incremental Property Checking • Compute A1 … A5 s.t. A1 U1 P A2 U2 A1 A3 U3 A2 A4 U4 A3 A5 U5 A4 true ARB A5 • Result P holds on U1 || .. || U5 || ARB • Two techniques • Recursive invocation • Interface specification • Comparison with non-compositional analysis
Analysis Results Results do not reflect the cost of generating the assumptions
Cost of Assumption Generation LTSA tool
Discussion • Significant memory savings • Serious time overhead • Experimented with SPIN’s feature for separate compilation • Encode assumptions as never claims • Restrict search of the verifier • New encoding for queries • Significant improvement • Cost of interface generation reduced from 818.213s to 185.185s
Conclusion and Future Work • Tool architecture for compositional verification • Uses L* for automatic derivation of assumptions • Automated derivation of interface specification • “Loose” integration with SPIN • Model checking for queries and conjectures • Application to Arbiter case study • Significant memory gains • Serious time overhead ; techniques to address this issue • Future work • Tighter integration with SPIN • Parallel checks for queries • Investigate buffered message passing communication • Liveness (learning for infinitary regular sets) • Distinguish between read and write operations
