1 / 45

Passive Testing

Passive Testing. Wissam Mallouli INT-LOR Wissam.mallouli@int-edu.eu Institut National des Télécommunications - Evry 11/30/2007. Table of contents. Introduction Active/Passive Testing Forward checking technique The backward checking technique Signature based technique Conclusion.

sarila
Télécharger la présentation

Passive Testing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Passive Testing Wissam Mallouli INT-LOR Wissam.mallouli@int-edu.eu Institut National des Télécommunications - Evry 11/30/2007

  2. Table of contents • Introduction • Active/Passive Testing • Forward checking technique • The backward checking technique • Signature based technique • Conclusion

  3. Table of contents • Introduction • Active/Passive Testing • Forward checking technique • The backward checking technique • Signature based technique • Conclusion

  4. Introduction • Fault detection : essential domain but fastidious. Need of automation. • Testing of a system. • Active/Passive testing techniques

  5. Table of contents • Introduction • Active/Passive Testing • Forward checking technique • The backward checking technique • Signature based technique • Conclusion

  6. Active Testing IUT Active Tester Verdict: PASS,FAIL, INCONC. Functional & Security Formal Specification Test Suites Automatic test generation based on formal descriptions Generation of a : - reasonable test scenarios number (Execution) - Complete (to cover all the system transitions)

  7. Conformance Testing(1/2) • Check if the implementation of a system conforms to its specification I O System (S) I O1=O??? System (I)

  8. Errors IMPLEMENTATION i1 / o2 1 2 SPECIFICATION output error i1 / o1 1 2 i1 / o1 1 3 transfert error i1 / o2 1 3 mixte error

  9. Passive Testing IUT Trace Collection Passive Tester PO Verdict: PASS,FAIL, INCONC. System User Security Properties Specification

  10. The two methods of conformity test (1/2) • Active testing : • principe : send messages through the implementation, wait for answer, compare answer with the desired one • need two phases : • test sequences generation • tests application • pros and cons : • + : possibility to focus on a particular area of the specification • - : automatic test generation is difficult • - : can disturb (or crash) the IUT

  11. The two methods of conformity test (2/2) • Passive testing : • principe : observation input/output sequences (trace), application of the trace on the spécification • pros and cons : • + : no interference with the IUT • + : no need of tests generation • - : algorithmsefficiency

  12. Table of contents • Introduction • Active/Passive Testing • Forward/backward checking technique • Signature based technique • Conclusion

  13. Principles • Trace + Formal specification of the system • To map the trace in the specification • 2 differents algorithms : forward and backward checking

  14. a/y b/y S1 S2 a/y b/y b/y a/x b/y S0 S3 a/x Example/Exercice a/x b/y a/x b/y b/y a/y b/y a/y b/y (trace 1) a/x b/y a/x b/y b/y a/x b/x a/y b/y (trace 2)

  15. EFSM • EFSM (Extended Finite State Machine) : • I/O events with or without parameters • a predicate (to be satisfied) • actions (to be done)

  16. EFSM : definitions • EFSM = heptuplet (S, s0, Sf, I, O, x, T) • S : finite non empty set of states • s0 : initial state • Sf : finite set of final states • I : finite set of input symbols (  parameters) • O : finite set of output symbols (  parameters) • x : finite set of variables • T : finite set of transitions

  17. EFSM • Transition = sextuplet (st, ft, it, ot, Pt, At) • st : initial state • ft : final state • it : input event • ot : output event • Pt : predicate • At : set of actions

  18. EFSM Formalism b/y a/y S0 S2 a/y P(X0) true b/y b/y A(X0) a/x S1 S3 b/y a/x S=(S0,S1,S2,S3) I=(a,b) O=(x,y) 18

  19. Variables values checking • Test by value determination • Test by determination of interval of variables

  20. Principle value determination according to an event trace (naive approach) Test by value determination

  21. Test by value determination • Algorithm in two phases : • phase 1 : homing. Trace exploration until only one candidat state stays • phase 2 : error detection. Trace exploration until the end, or an error

  22. Test by value determination • Variables determination : • few possibles transitions giving different values to one variable  UNDEFINED variable • predicate with at least one UNDEFINED variable  accepted predicate (doubt)

  23. State of art of passive testing • Test by value determination • Test by determination of interval of variables

  24. Test by determination of interval of variables • Principle • Intervals to record values • Assertions = boolean formula of predicates • Uses Candidate Configuration Set (CCS)

  25. Le test par détermination d'intervalles de valeurs • Intervalles • objectif : résoudre le problème de la perte d'information (plusieurs valeurs possibles pour une variable) • notation : • a v  b  R(v) = [a; b] • v = a  R(v) = [a; a]

  26. Test by determination of interval of variables • Intervals : operations • sum of two intervals : • [a; b] + [c; d] = [a+c; b+d] • Subtraction of two intervals : • [a; b] - [c; d] = [a-c; d-b] • multiplication of an interval by a scalar : • w  [a; b] = [wa; wb] si w  0 • w  [a; b] = [wb; wa] si w  0

  27. Test by determination of interval of variables • Consistency and refinement test • refinement of intervals according to constraints (predicats and actions) • if an interval becomes empty then the configuration is false

  28. Table of contents • Introduction • Error types and testing methods • State of art of passive testing • The backward checking technique • Conclusion

  29. The backward checking technique • Backward checking : principle • Backtracking a transition • Backtracking a trace • Backtracking the past of a trace

  30. Backward checking : principle • Idea : « To understand present we must study the past »

  31. Backward checking : principle • Backward checking : • backtracking the event trace • backtracking the past of the trace past of the trace FAIL FAIL trace FAIL OK sens of the trace FAIL sens of the algorithm

  32. Backward checking : principle • Advantages of this approach : • start with correct information • build artificially a longer trace

  33. Backward checking : principle • To validate a trace in the past we must validate special variables : determinants • Definition : a variable v is a determinant for the trace t if v must necessarily be validated before we can validate t

  34. The backward checking technique • Backward checking : principle • Backtracking a transition • Backtracking a trace • Backtracking the past of a trace

  35. 4 f i Backtracking a transition P : u>=1 A : x=1 y=y+1 z=v+w R = < f ; u [0;3], x [1;1], y [2;8], z [1;2], a [7;9] ; Asrt = - ; D = {u, y, z, a} > R = < i ; u [1;3], a [7;9], cste [1;2], y [1;7] ; Asrt = v+w=cste ; D = {u, a, y, v, w} >

  36. The backward checking technique • Backward checking : principle • Backtracking a transition • Backtracking a trace • Backtracking the past of a trace

  37. Backtracking a trace • similar to intervals determination algorithm but backward • process the transition backtracking for each I/O couple

  38. The backward checking technique • Backward checking : principle • Backtracking a transition • Backtracking a trace • Backtracking the past of a trace

  39. Table of contents • Introduction • Active/Passive Testing • Forward checking technique • The backward checking technique • Invariant based technique • Conclusion

  40. Test by invariants : principle • Definition : invariant = property that is always true • Two step test : • extraction of invariants from the specification • application of invariants on event traces from implementation • Problem : temporal logic not in the model • Solution : I/O invariants

  41. Test by invariants : I/O invariants • An invariant is composed of two parts : • the test (an input or output) • the preambule (I/O sequence) • 3 kind of invariants : • output invariants • input invariants • succession invariants

  42. Test by invariants : Output invariant • Definition : invariant in which the test is an output • Meaning : « immediatly after the sequence préambule there is always the output test • Example : (i1 / o1) (i2 / o2) (preambule in blue, test in red)

  43. Test by invariants : Input invariant • Definition : invariant in which the test is an input • Meaning : « immediatly before the sequence préambule there is always the input test • Example : (i1 / o1) (i2 / o2) (i3/ (preambule in blue, test in red)

  44. Test by invariants : succession invariant • Definition : I/O invariant for complex properties (loops …) • Example : • the 3 invariants below build the property : « only the third i2 we meet is followed by o3 » (i1 / o1) (i2 / o2) (i1 / o1) (i2 / o2) (i2 / o2) (i1 / o1) (i2 / o2) (i2 / o2) (i2 / o3)

  45. Test by invariants • + : powerful method (complex properties) • + : fast when invariants are known • - : difficulties to extract interesting invariants (not automated) • - : sensible to non-determinism of specification • - : doesn't detect every errors

More Related