Going with the FlowParameterized Verification using Message Flows Murali Talupur & Mark Tuttle SCL, Intel Corporation
Protocol Verification • Distributed protocols are crucial components of modern computer systems • Cache coherence protocols for example • Designed parametrically • Full validation requires parametric reasoning • Protocol verification involves two main issues • Tightly coded • So standard predicate abstraction and COI reduction will not work • Unbounded or high degree of parallelism
Our Approach Consists of two key elements • CMP Method: • A general framework for reasoning about systems with • replication • We simplified and generalized the method • Flow based Invariants: • A new method for discovering invariants of concurrent • systems • Implicit partial orders on system events yield valuable • invariants
Existing Methods Regular Model Checking Aggregated Trans Counter Abstraction Theorem Proving Invisible Invariants CMP Increasing Manual Effort WS1S Index predicates Automatic methods don’t scale Manual methods require human guidance but scale
CMP method • Compositional reasoning based method • Proposed by McMillan, elaborated by Chou et al [CTC] and further formalized by Krstic • CMP: Pros and Cons • Scales to large protocols • This was one of the first techniques to handle Flash protocol • Within Intel handled a cache protocol several orders of magnitude bigger than even Flash • User has to supply “lemmas”/invariants • Supplying lemmas is easier than supplying full blown inductive invariants • Easier than pure theorem proving
CMP Method True or Real Cex P(N) PA Abstract Model Check spurious cex P#(N) Strengthen Invent Lemma
Data Type Reduction • Reduces unbounded range [1..N] to [1,2, o] • Throws away the state spaces of [3..N] • Any condition involving them is conservatively over-approximated 1 2 PA Other 1 2 3 N-1 N P(N)
ruleset src : [1..N] do rule “Local_Get" UniMsg[src].Cmd = Get & !Dir.Pending ==> begin Dir.Dirty := false; UniMsg[src].Cmd := Put; endrule; endruleset; ruleset src : [1,2] do rule “Local_Get" UniMsg[src].Cmd = Get & !Dir.Pending ==> begin Dir.Dirty := false; UniMsg[src].Cmd := Put; endrule; endruleset; rule "ABSLocal_Get" true & !Dir.Pending ==> begin Dir.Dirty := false; NOP endrule; Rules are in guarded command form: r: ! a
Data Type Reduction • Data type reduction is syntactic • Very fast • Abstract model has small state space • Behavior of “Other” is not constrained at all • Need to refine “Other”
Refinement User provides relevant lemmas Proceeds in two steps: • Strengthening of P(N) with the lemmas • Abstracting the strengthened system
ruleset src : NODE do rule “Local_Get" UniMsg[src].Cmd = Get & !Dir.Pending ==> begin Dir.Dirty := false; UniMsg[src].Cmd := Put; endrule; endruleset; ruleset src : NODE do rule "Local_Get" UniMsg[src].Cmd = Get & !Dir.Pending & forall dst: NODE do src != dst -> !(Proc[dst].State = Excl) end ==> begin Dir.Dirty := false; UniMsg[src].Cmd := Put; endrule; endruleset; Strengthening invariant "Lemma" forall src : NODE do forall dst : NODE do dst != src -> (UniMsg[src].Cmd = Get -> !(Proc[dst].State = Excl))
ruleset src : NODE do rule "Local_Get" UniMsg[src].Cmd = Get & !Dir.Pending & forall dst: [1..N] do src != dst -> !(Proc[dst].State = Excl) end ==> begin Dir.Dirty := false; UniMsg[src].Cmd := Put; endrule; endruleset; rule "ABSLocal_Get" true & !Dir.Pending & forall dst: [1,2]. -> !(Proc[dst].State = Excl) end ==> begin Dir.Dirty := false; NOP endrule; Refinement Abstracting strengthened system leads to a refined abstract model
CMP Method Conjunction of property to be proved and lemmas 8 i,j. (i,j) P(N) ² P(N) ² (1,2) Strengthen with 8 i,j. (i,j) Apparent Circular Reasoning Paper has the proof details! Ps(N) ² (1,2) Abstraction DTR is conservative PA ² (1,2)
Flow based Invariants • Rich partial orders on system events are implicit in the concurrent protocols • We call these Message Flows or simply Flows • And these can yield powerful invariants
Flows: Examples Dir i j Dir i j SendReqS SendReqS RecvReqS RecvReqS SendInv SendGntS SendInvAck RecvGntS RecvInvAck SendGntS RecvGntS Process i intiates a Request Shared transaction: Case 1 Process i intiates a Request Shared transaction: Case 2
Constraints from Flows Dir i j SendReqS ReqShare(i) SendReqS(i),RecvReqS(i),SendGntS(i),RecvGntS(i) RecvReqS Precedence between rules: For instance, for process i, action RecvReqS(i) must happen before SendGntS(i) SendGntS RecvGntS
Tracking Flows A set Aux(i) of auxiliary variables to track 1) all the flows that a process i is involved in 2) for each such flow the last rule that was fired Each aux 2 Aux(i) is initially ( no_flow, no_rule ) If process i fires rules rn in a flow f: r1,….rn-1, rn,…..rm update aux = (f,rn-1) to (f,rn) If rn is the last rule reset the aux variable
Constraints from Flows Dir i j SendReqS Precondition for process i firing rn in flow f: C(rn,f): 9 aux 2 Aux(i). aux = (f, rn-1 ) RecvReqS SendGntS RecvGntS
Lemmas from Flows Let r: ! a appear in flows f1,..,fn Find precedence constraints C1(r,f1), ..,Cn(r,fn) Let pre(r) = C1(r,f1) Ç ..Ç Cn(r,fn) Lemma: ) pre(r)
invariant "Lemma_1" forall i : NODE do Chan3[i].Cmd = InvAck & CurCmd != Empty & ExGntd = true -> Chan3[i].Data = AuxData & forall j : NODE do j != i -> Cache[j].State != E & Chan2[j].Cmd != GntE & Chan3[j].Cmd != InvAck Deriving Lemmas from Flows • We can easily convert message flows into invariants • Our experiments indicate the resulting invariants are very powerful • Advantages: • Message flows are readily available in design documents • Easy to understand • Flows are local and linear • No low level details of the protocol • Valuable validation collateral A German Flow ReqShare: SendReqS,RecvReqS,SendInval,SendGntS,RecvGntS
Design Doc Modified CMP Flow Flows Done or Real Cex Pa(N) Add aux and Flow lemmas PA Abstract Model Check P(N) spurious cex P#(N) Strengthen Invent Lemma
Experimental Results • We applied our method to two standard examples • German and Flash • Verified both control and data property • Compared against CTC • German • 3 flows and 2 lemmas added manually • Lemmas slightly simpler than ones in CTC • Flash • 6 flows and 2 lemmas in contrast to > 5 lemmas in CTC • one trivial and one from CTC paper • Lemma complexity reduced by 75% • One auxiliary variable in contrast to 4 in CTC paper • Different from flow auxiliary variables • This helps our running times as well: couple of minutes compared to couple of hours in the CTC work
Conclusion/ Future Work • A powerful new method for parameterized verification based on • the CMP method and • invariants from Flows • CMP Method: • (i) We have simplified and generalized the CMP method • (ii) Allows more abstractions than simple data type reduction • Thus, more widely applicable • Flows: • A new way to derive invariants for concurrent systems • Look for implicit partial orders on system events!
Conclusion/Future Work More can be done with flows: Online monitoring to see all traces conform to flows Extend flows to other distributed systems Flows for shared memory systems using other types of system events… Learn flows automatically…
