html5-img
1 / 75

Implementation and Testing of Distributed Systems

Implementation and Testing of Distributed Systems. R. M. Hierons Brunel University, UK rob.hierons@brunel.ac.uk http://people.brunel.ac.uk/~csstrmh. Work With. Jessica Chen Mercedes Merayo Manuel Nunez Hasan Ural. tester. tester. tester. Multi-port systems.

suzy
Télécharger la présentation

Implementation and Testing of Distributed Systems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Implementation and Testing of Distributed Systems R. M. Hierons Brunel University, UK rob.hierons@brunel.ac.uk http://people.brunel.ac.uk/~csstrmh MTVV 2009

  2. Work With • Jessica Chen • Mercedes Merayo • Manuel Nunez • Hasan Ural MTVV 2009

  3. tester tester tester Multi-port systems • Physically distributed interfaces/ports • We place a tester at each port SUT MTVV 2009

  4. Context/assumptions • Testing is black-box: we do not have access to anything ‘inside’ the system. • Communications is synchronous or there is a ‘slow environment’. • Specification is complete. • The system under test behaves like an unknown model that can be represented using the same formalism as the specification. MTVV 2009

  5. Motivation • Initially just testing – and testing in the distributed test architecture. • The discussion will be around both • testing and • implementation/conformance relations. MTVV 2009

  6. Structure • Background • Testing from: • deterministic finite state machines • nondeterministic finite state machines • input output transition systems MTVV 2009

  7. Background MTVV 2009

  8. Global Traces • A global trace is an input/output sequence • We assume there are m ports and let: • X be the input alphabet/set divided into X1, …, Xm: xi will denote an input from Xi • Y be the output alphabet, each element being in (Y1{-})…(Ym{-}): yi will denote an element of Yi • A global trace is an element of (X  Y)* • The set of global traces of a system can be seen as its behaviour. MTVV 2009

  9. x1 y1 y2 x1 y1 Example • Global trace x1/(y1,y2) x1/(y1,-) MTVV 2009

  10. The distributed test architecture • A tester at each port • Testers cannot communicate with one another • No global clock • Advantages: • Simple and cheap to implement • Might represent expected usage MTVV 2009

  11. Tester 1 SUT Tester 2 x1 y1 y2 x1 y1 Consequences • Each tester observes only the interactions at its port – we cannot observe global traces • The tester at port 1 observes x1y1x1y1 and the tester at 2 observes y2 only. MTVV 2009

  12. Local Traces • At a port p we observe a local trace: an element of (Xp  Yp)* • Given global trace z(X  Y)* and port p we let p(z) be the projection of z onto p. • A local trace is observed at each port. • We are interested in the case where either: • Testing can only observe local traces; or • Users only observe local traces. MTVV 2009

  13. Two possibilities • We might have: • Agents at ports are entirely ‘independent’: • No external agent can receive information regarding observations at more than one port • Or the local traces observed at the ports can be ‘brought together’ later. MTVV 2009

  14. Differences • Specification: • Response to x can either be (y,z) or (y’,z’) [at ports 1,2] • Implementation • Response to x is (y,z’) • Mostly we will assume that local traces can be brought together later. MTVV 2009

  15. tester SUT tester Controllability problems • The following test has controllability problems: introduces nondeterminism into testing. MTVV 2009

  16. tester SUT tester tester SUT tester x1 x1 y1 y1 y2 x1 x1 y2 y1 y1 Observability problems • The following look the same • Testers/users cannot ‘map’ output to input MTVV 2009

  17. Equivalent global traces • Since we only observe local traces: • Global traces z and z’ are indistinguishable if their projections are identical: the local traces are the same. • We denote this: zz’ • The following are equivalent under  • x1/(y1,y2)x1/(y1,-) • x1/(y1,-)x1/(y1, y2) • Both have x1y1x1y1 at port 1 and y2 at 2. MTVV 2009

  18. Local verdicts • Normally a test case ends in a state that denotes the verdict of the test run. • We have a tester at each port – we could include ‘local verdicts’ in each tester. • The overall verdict is pass if and only if all ‘local verdicts’ are pass. • Here: • local traces cannot be brought together • Normally this is not sufficient. MTVV 2009

  19. tester SUT tester tester Spec tester x1 x1 y1 y1 y2 x1 x1 y2 y1 y1 Observation: Test effectiveness is not monotonic • Example: x1 detects a fault but x1x1 does not. MTVV 2009

  20. Using an external network • If we connect the testers using an external network, sometimes we can: • For controllability: • If we want input x at port p to be followed by input x’ at port q, the tester at p can send a coordination message after it has sent x. • For observability: • Testers send coordination messages to ‘divide up’ the traces observed at each port. MTVV 2009

  21. But • If a system has physically distributed interfaces then the implementation relation should reflect this: • Even if we can connect the testers, we should be careful that we do not give the verdict fail when the behaviour is acceptable in use. • The users will only observe local traces. MTVV 2009

  22. Past research • Almost all has focussed on testing from a deterministic finite state machine (DFSM). Two main topics of interest: • Generating test sequences that do not suffer from controllability and/or observability problems • Adding coordination messages (possibly adding a minimum number). MTVV 2009

  23. Problems/issues • A DFSM can have transitions that cannot be executed without controllability problems. • ‘Complete’ test generation algorithms place conditions on the DFSM – they are not general. • The methods test against the ‘traditional’ implementation relation – aiming to do too much? MTVV 2009

  24. The solution • We need a good understanding of what it means to distinguish two models with distributed ports. • We have seen that this gives us new implementation relations. • Then we want to test against these. MTVV 2009

  25. Deterministic Finite State Machines MTVV 2009

  26. The behaviour of M in state si is defined by the set of input/output sequences (traces) from si s2 s3 a/0 b/0 b/1 s5 b/1 a/1 a/0 a/1 b/0 s1 s4 b/1 a/0 Finite State Machines MTVV 2009

  27. Implementation relations • For single-port deterministic finite state machines (DFSMs) say that: • N conforms to M if and only if N and M are equivalent (they define the same sets of global traces). • We can express this in terms of their initial states being equivalent. • If M is nondeterministic we require that N is a reduction of M: • Every trace of N is a trace of M. MTVV 2009

  28. Properties • Classical DFSM conformance is just DFSM equivalence. • It is an equivalence relation. • We have an associated notion of minimality: • A DFSM M is minimal if there is no smaller DFSM that is equivalent to M • For a DFSM M there is a unique equivalent minimal DFSM. MTVV 2009

  29. An implementation relation for distributed systems • We say that DFSM N conforms to DFSM M if: • Every global trace of N is indistinguishable from a global trace of M. • Equivalently: • For every global trace z of N there is a global trace z’ of M such that z  z’. MTVV 2009

  30. x1/(y1,-) s2 x2/(-, y2) x1/(y1,-) x1/(y1,-) s1 s1 x2/(-, y’2) x1/(y1,-) x2/(-, y’2) s3 x2/(-, y’2) Conformance is weaker than equivalence • This also shows that it is not an equivalence relation (second has behaviours inconsistent with first). • Is the first an acceptable design for second? Conforms to MTVV 2009

  31. Observation • There are many notions of equivalence in the LTS/IOTS literature. • Have we reinvented one of these? • Previous notions of equivalence reduce to isomorphism when considering minimal, completely-specified DFSMs. • Local equivalence does not. MTVV 2009

  32. FSM M FSM M’ Conforms (old) Conforms (new) Implementation N Refinement and testing • It appears that we must produce new refinement rules and new test techniques. • Alternatively (if we can observe global traces in testing): MTVV 2009

  33. x1/(y1,-) s2 x2/(-, y2) x1/(y1,-) s1 x1/(y1,-) x2/(-, y’2) s3 x2/(-, y’2) Example • Here we transform to a non-deterministic FSM x1/(y1,-) s2 x2/(-, y2) x1/(y1,-) x1/(y1,-) s1 x1/(y1,-) x2/(-, y’2) s3 x2/(-, y’2) MTVV 2009

  34. x1/(-,-) s1 s2 x2/(y’1, y’2) x1/(y1, y2) x2/(-, -) x1/(y1, y2) x1/(y’1, y’2) s3 s4 x2/(y1, y2) x2/(y’1, y’2) Not a general solution • The language of words that are equivalent under  is not regular (intersect with X*) MTVV 2009

  35. Controllable testing MTVV 2009

  36. tester SUT tester Controllability problems • The following test has a controllability problem. MTVV 2009

  37. Controllability for DFSM • An input sequence x=x1…xk is controllable if the corresponding trace z=x1/y1…xk/yk has the following property: • Every xi, i>1, is applied at a port pi such that xi-1/yi-1 has either input or output at pi. • This means: • The tester at pi knows when to apply xi. • Ideally we wish to restrict testing to controllable sequences • Initial work focussed on this case. MTVV 2009

  38. Distinguishing states • If we restrict ourselves to controllable testing we need: • x causes no controllability problems from s and s’ • x leads to different sequences of interactions, for s and s’, at some port. • We say that x locally s-distinguishes s and s’. • If no input sequence locally distinguishes s and s’ they are locally s-equivalent. MTVV 2009

  39. x2/(-, y2) x1/(y1,-) s1 s2 x1/(y1,-) x2/(y1, y2) x2/(y1,-) x2/(-, y2) x1/(y1,-) s4 s3 x1/(y1,-) Testing is weaker • We cannot locally s-distinguish s1 and s4 but x1x2 locally distinguishes them. MTVV 2009

  40. Distinguishing two states • Given port p and states s1 and s2 of m-port FSM M with n states: • s1 and s2 are locally s-distinguishable by an input sequence starting at p if and only if they are locally s-distinguished by some such input sequence of length at most m(n-1). • This bound is ‘tight’. • The sequences can be found in low-order polynomial time. MTVV 2009

  41. Distinguishing all states • A ‘complete’ set of input sequences that locally s-distinguish the locally s-distinguishable states of M can be found in O(pn2), where p is the size of the input alphabet. • This has at most n-1 sequences, each of length at most m(n-1) MTVV 2009

  42. Minimality • Two possible definitions: • Def 1: A DFSM is locally s-minimal if it has no locally s-equivalent states. • Def 2: A DFSM M is locally s-minimal if no DFSM with fewer states is locally s-equivalent to M. • For initially-connected, completely specified, single-port DFSMs, these are the same. MTVV 2009

  43. x2/(-, y2) x1/(y1,-) s1 s2 x1/(y1,-) x2/(y1, y2) x2/(y1,-) x2/(-, y2) x1/(y1,-) s4 s3 x1/(y1,-) Minimal DFSMs are not always locally s-minimal • We have seen that s1 and s4 are locally s-equivalent MTVV 2009

  44. x2/(-, y2) x2/(-, y2) x1/(y1,-) x1/(y1,-) s1 s2 s1 s2 x1/(y1,-) x1/(y1,-) x2/(-, y2) x2/(-, y2) x2/(y1,-) x1/(y1,-) x2/(y1,-) x2/(-, y2) x1/(y1,-) s4 s3 s3 x1/(y1,-) Merging s-equivalent states • A smaller acceptable design? MTVV 2009

  45. x2/(-, y2) x1/(y1,-) x2/(-, y2) x1/(y1,-) s1 s1 s2 x1/(y1,-) x2/(-, y2) x2/(y1,-) x2/(-, y2) x1/(y1,-) s4 s3 x1/(y1,-) Minimising: smallest FSM • Even smaller: MTVV 2009

  46. Consequences • We had two alternative definitions. • Def 1: A DFSM is locally s-minimal if it has no locally s-equivalent states. • Def 2: A DFSM M is locally s-minimal if no DFSM with fewer states is locally s-equivalent to M. • For multi-port DFSMs these differ. • Def 2 is ‘better’? MTVV 2009

  47. Canonical FSMs • Given DFSM M, we can find: • Maximal Mmax that is locally s-equivalent to M • Minimal Mmin that is locally s-equivalent to M • We can find them efficiently. MTVV 2009

  48. Results • DFSM N is locally s-equivalent to DFSM M if and only if N is a reduction of Mmax. • The set of DFSMs that are locally s-equivalent to a DFSM M forms a bounded lattice. MTVV 2009

  49. FSM M FSM Mmax reduction s-equivalence Implementation N Refinement and testing • We now know that: MTVV 2009

  50. Open questions • How do we ‘target’ local equivalence or local s-equivalence in test generation? • When can we produce equivalent of Mmax for local equivalence? • Generating locally minimal/s-minimal DFSMs? Second problem is NP-hard but is polynomial if we have two ports. MTVV 2009

More Related