1 / 98

Generated Path Conditions for Timed Systems

Generated Path Conditions for Timed Systems. Doron Peled Dept. of Computer Science University of Warwick United Kingdom Joint work with Saddek Bensalem , Hongyang Qu, Stavros Tripakis. IFM 2005. Tester ’ s Goals. Help in selecting test cases. Visual, by clicking on a path in flow chart.

leia
Télécharger la présentation

Generated Path Conditions for Timed Systems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Generated Path Conditions for Timed Systems Doron Peled Dept. of Computer ScienceUniversity of Warwick United Kingdom Joint work with Saddek Bensalem, Hongyang Qu,Stavros Tripakis IFM 2005 IFM 2005, Eindhoven

  2. Tester’s Goals • Help in selecting test cases. • Visual, by clicking on a path in flow chart. • According to intuition about potential errors. • According to some formal specification. • Performing tests • Forcing an execution (even when nondeterminism exists). • Calculating the probability of a path. IFM 2005, Eindhoven

  3. Architecture Test Selector Executor Visual Selection Model Checker Add Synchro. transitions Counter-exampletest case code compiler Calculate Weakest Precondition SatSolver Flow graph Calculate Probability IFM 2005, Eindhoven

  4. Unit testing: Selection of test cases (for white-box testing) The main problem is to select a good coverage criterion. Some standard options are: • Cover all paths of the program. • Execute every statement at least once. • Each decision (diamond node on flow chart) has a true or false value at least once. • Each condition predicate is taking each truth value at least once. • Check all possible combinations of conditions in each decision. IFM 2005, Eindhoven

  5. How to cover the executions? if (A>1)&(B=0) then X=X/A; if (A=2)|(X>1) then X=X+1; • Choose values for A,B,X at the beginning that would force the right path/conditions/predicates. • Value of X may change, depending on A,B. • What do we want to cover? Paths? Statements? Conditions? IFM 2005, Eindhoven

  6. By choosing A=2,B=0,X=3 each statement will be chosen. The case where the tests fail is not checked! if (A>1)&(B=0) then X=X/A; if (A=2)|(X>1) then X=X+1; Statement coverageExecute every statement at least once Now x=1.5 IFM 2005, Eindhoven

  7. Can be achieved using A=3,B=0,X=3 A=2,B=1,X=1 Problem: Does not test individual predicates. E.g., when X>1 is erroneous in second decision. if (A>1)&(B=0) then X=X/A; if (A=2)|(X>1) then X=X+1; Decision coverageEach decision (diamond node in flow graph) tested with true and false outcome at least once. IFM 2005, Eindhoven

  8. Preliminary:Relativizing assertions A (B) : x1= y1 * x2 + y2 /\ y2 >= 0 Relativize B) w.r.t. the assignment becomes B) [Y\g(X,Y)] (I.e., (B) expressed w.r.t. variables at A.)  (B)A =x1=0 * x2 + x1 /\ x1>=0 Think about two sets of variables,before={x, y, z, …} after={x’,y’,z’…}. Rewrite (B) using after, and the assignment as a relation between the set of variables. Then eliminate after by substitution. Here: x1’=y1’ * x2’ + y2’ /\ y2’>=0 /\x1=x1’ /\ x2=x2’ /\ y1’=0 /\ y2’=x1now eliminate x1’, x2’, y1’, y2’. Y=g(X,Y) (y1,y2)=(0,x1) B A (y1,y2)=(0,x1) B IFM 2005, Eindhoven

  9. Verification conditions: tests B T F C) is transformed toB)= t(X,Y) /\C) D) is transformed toB)=t(X,Y) /\ D) B)= D) /\ y2x2 t(X,Y) C D B T F y2>=x2 C D IFM 2005, Eindhoven

  10. How to find values for coverage? • Put true at end of path. • Propagate path backwards. • On assignment, relativize expression. • On “yes” edge of decision node, add decision as conjunction. • On “no” edge, add negation of decision as conjunction. • Can be more specific when calculating condition with multiple condition coverage. yes A>1/\B=0 no X=X/A yes A=2\/X>1 true no X=X+1 true IFM 2005, Eindhoven

  11. How to find values for coverage? (A2 /\ X/A>1) /\ (A>1 & B=0) yes A>1/\B=0 A2 /\X/A>1 no Need to find a satisfying assignment: A=3, X=6, B=0 Can also calculate path condition forwards. X=X/A A  2/\X>1 yes A=2\/X>1 true no X=X+1 true IFM 2005, Eindhoven

  12. Some real life story • An expert programmer inspects the code of NASA MER. • He observes through his experience and intuition that some execution path is suspicious. • He decides how to force this path to execute, e.g., by figuring some inputs and initial values. • He executes the path, showing his supervisor the presence of an error. • We want to build some tools to help him with this process. • We’ll use LTL to help with formalizing the intuition on where the error is. IFM 2005, Eindhoven

  13. Learning from another technique: Model Checking • Automaton description of a system B. • LTL formula . Translate  into an automaton P. • Check whether L(B)  L(P)=. • If so, S satisfies . Otherwise, the intersection includes a counterexample. • Repeat for different properties. ¬  IFM 2005, Eindhoven

  14. Unit Checking Unit Testing Model Checking IFM 2005, Eindhoven

  15. New: Test case generation based on LTL specification LTLAut Model Checker Path Path condition calculation Flowchart Compiler Transitions First order instantiator Test monitoring IFM 2005, Eindhoven

  16. Path conditions • Path in flow chart  multiple executions following path. • First order formula. • All executions of a path must start with initial values satisfying the path condition. • In deterministic code, there can be only one execution starting with particular values, hence all executions starting with initial values satisfying the path condition will follow that path. • In nondeterministic code, each such initial value has an execution following a path. May need to insert synchronizing code. IFM 2005, Eindhoven

  17. Goals • Verification of software. • Compositional verification. Use only a unit of code instead of the whole code. • Parameterized verification. Verifies a procedure with any value of parameters in “one shot” • Generating test cases via path conditions: A truth assignment satisfying the path condition. Helps derive the demonstration of errors. • Generating appropriate values to missing parameters. IFM 2005, Eindhoven

  18. ¬at l2 at l2/\ xy ¬at l2 at l2/\ x2y Spec: ¬at l2U (at l2/\ xy /\ (¬at l2/\(¬at l2U at l2 /\ x2y ))) Observation:each node hasconjunctions of predicates onprogram variables and programcounters • Automatic translation of LTL formula into an automaton [Gerth et all] • LTL is interpreted over finite sequences. • Can use other (linear) specification. • Property specifies the path we want to find (SPIN: never claim),not the property that must hold for all paths (for this, take the negation). IFM 2005, Eindhoven

  19. Divide and Conquer • Intersect property automaton with theflow chart, regardless of the statements and program variables expressions. • Add assertions from the property automaton to further restrict the path condition. • Calculate path conditions for sequences found in the intersection. • Calculate path conditions on-the-fly. Backtrack when condition isfalse.Thus, advantage to forward calculation of path conditions (incrementally). IFM 2005, Eindhoven

  20. l2:x:=x+z l3:x<t l1:… Spec: (only program counters here)¬at l2U (at l2/\¬at l2/\(¬at l2U at l2)) at l2 l2:x:=x+z ¬at l2 X = ¬at l2 at l2 l3:x<t ¬at l2 at l2 l2:x:=x+z Either allexecutions of a path satisfy the formula or none. at l2 Sifts away paths not satisfying formula. Then calculate path condition. IFM 2005, Eindhoven

  21. l2:x:=x+z l3:x<t l1:… Spec: ¬at l2U (at l2/\ xy /\ (¬at l2/\(¬at l2U at l2 /\ x2y ))) xy l2:x:=x+z ¬at l2 X = at l2/\ xy l3:x<t x2y ¬at l2 l2:x:=x+z Only some executions of path may satisfy formula at l2/\ x2y Modify calculation of path condition to incorporate property IFM 2005, Eindhoven

  22. Calculating the intersection of the property automaton and flow graph (abstract variables away). a a a <>a s1 s2 q1 q2 a s3 ¬a a a Acceptance isdetermined bypropertyautomaton. s2,q1 s1,q1 s1,q2 s3,q2 ¬a IFM 2005, Eindhoven

  23. How to generate test cases • Take the intersection of an LTL automaton (for a never claim) with the flow graph. Some paths would be eliminated for not satisfying the assertions on the program counters. • Seeing same flow chart node does not mean a loop: program variables may value. Use iterative deepening. • For each initial path calculate the path condition. Backtrack if condition simplifies to false. • Report path condition based on flow graph path+LTL assertions. • Always simplify conditions! IFM 2005, Eindhoven

  24. How the LTL formula directs the search Consider (x=4)U (x=5/\o…) x=4 x<5 false true x=5 y:=7 x:=x+1 IFM 2005, Eindhoven

  25. How the LTL formula directs the search Consider x=4U (x=5/\o…) x=4 x<5 false true x=5 y:=7 x:=x+1 IFM 2005, Eindhoven

  26. How the LTL formula directs the search Consider x=4U (x=5/\o…) x=4 X=4 x<5 false true x=5 y:=7 x:=x+1 IFM 2005, Eindhoven

  27. How the LTL formula directs the search Consider x=4U (x=5/\o…) x=4 X=4 x<5 false true x=5 X=4 y:=7 x:=x+1 IFM 2005, Eindhoven

  28. How the LTL formula directs the search Consider x=4U (x=5/\o…) x=4 X=4 x<5 false true x=5 X=4 This is acontradiction y:=7 x:=x+1 X=4 x<5 true IFM 2005, Eindhoven

  29. How the LTL formula directs the search Consider x=4U (x=5/\o…) x=4 X=5 x<5 false true x=5 X=4 y:=7 x:=x+1 IFM 2005, Eindhoven

  30. How the LTL formula directs the search Consider x=4U (x=5/\o…) x=4 X=5 x<5 false true x=5 X=4 y:=7 x:=x+1 IFM 2005, Eindhoven

  31. l0 Example: GCD l1:x:=a l2:y:=b l3:z:=x rem y l4:x:=y l5:y:=z no l6:z=0? yes l7 IFM 2005, Eindhoven

  32. l0 Example: GCD l1:x:=a l2:y:=b Oops…with an error (l4 and l5 were switched). l3:z:=x rem y l4:y:=z l5:x:=y no l6:z=0? yes l7 IFM 2005, Eindhoven

  33. Why use Temporal specification • Temporal specification for sequential software? • Deadlock? Liveness? – No! • Captures the tester’s intuition about the location of an error:“I think a problem may occur when the program runs through the main while loop twice, then the if condition holds, while t>17.” IFM 2005, Eindhoven

  34. l0 Example: GCD l1:x:=a l2:y:=b a>0/\b>0/\at l0 /\at l7 l3:z:=x rem y at l0/\a>0/\b>0 l4:y:=z l5:x:=y no l6:z=0? yes at l7 l7 IFM 2005, Eindhoven

  35. l0 Example: GCD l1:x:=a l2:y:=b a>0/\b>0/\at l0/\at l7 l3:z:=x rem y Path 1: l0l1l2l3l4l5l6l7a>0/\b>0/\a rem b=0 Path 2: l0l1l2l3l4l5l6l3l4l5l6l7a>0/\b>0/\a rem b0 l4:y:=z l5:x:=y no l6:z=0? yes l7 IFM 2005, Eindhoven

  36. Potential explosion Bad point: potential explosion Good point: may be chopped on-the-fly IFM 2005, Eindhoven

  37. Now we add time • Detailed model, for each transition we have 4 parameters [l, u, L, U]: • l Needs to be enabled at least that much. • u Cannot be enabled without taken longer than that. • L Least time for transformation to occur (after been chosen). • U Transformation cannot take more than that. IFM 2005, Eindhoven

  38. Translation to timed automata s1at l c1c2x2:=0 c1c2x2:=0 c1c2x1:=0 c1c2x1:=0 c1c2x1,x2:=0 c1c2 c1c2 c1c2 s3,at lx2<u2x1<u1 c1c2 c1c2 s2,at lx1<u1 s4,at lx2<u2 c1c2x2:=0 c1c2x1:=0 Timing out the enabledness:Zero the counters,Cannot wait enabled too much. IFM 2005, Eindhoven

  39. Translation to timed automata s1at l c1c2x2:=0 c1c2x2:=0 c1c2x1:=0 c1c2x1:=0 c1c2x1,x2:=0 c1c2 c1c2 c1c2 s3,at lx2<u2x1<u1 c1c2 c1c2 s2,at lx1<u1 s4,at lx2<u2 c1c2x2:=0 c1c2x1:=0 ac x1l1x1:=0 bc x1l1x1:=0 x2l2x2:=0 x2l2x2:=0 bc ac s5x1<U1 s6x2<U2 Can fire only if waited enough,Zero counters again. IFM 2005, Eindhoven

  40. Translation to timed automata s1at l c1c2x2:=0 c1c2x2:=0 c1c2x1:=0 c1c2x1:=0 c1c2x1,x2:=0 c1c2 c1c2 c1c2 s3,at lx2<u2x1<u1 c1c2 c1c2 s2,at lx1<u1 s4,at lx2<u2 c1c2x2:=0 c1c2x1:=0 ac x1l1x1:=0 bc x1l1x1:=0 x2l2x2:=0 x2l2x2:=0 bc ac s5x1<U1 s6x2<U2 Conditions on paths represented using (symbolic) DBMs. x1L1 x2L2 af bf s8 s7 IFM 2005, Eindhoven

  41. Its easy to select an interleaved sequence. But due to time limitations, it may execute in a different order. Just the order on events from the same process and using same variables is to be considered. Should we really look at paths? a a b b c d c Samevariable d Sameprocess IFM 2005, Eindhoven

  42. Generate an automaton for all consistent interleavings b a a b c d a b c Intersect this automaton with automaton for system.Calculate “partial order” condition: start from leaves.When there is a choice, usedisjunct. d c b d c IFM 2005, Eindhoven

  43. Generate an automaton for all consistent interleavings b a a b c d c b d c IFM 2005, Eindhoven

  44. Generate an automaton for all consistent interleavings b a a b c d c b d c IFM 2005, Eindhoven

  45. Generate an automaton for all consistent interleavings b a a b c d c b d c IFM 2005, Eindhoven

  46. An example — a simple network protocol IFM 2005, Eindhoven

  47. The flow charts IFM 2005, Eindhoven

  48. Path — no timeout IFM 2005, Eindhoven

  49. Precondition The simplified precondition: l >= 110 IFM 2005, Eindhoven

  50. The diagrams IFM 2005, Eindhoven

More Related