Doron Peled, Bar Ilan University

1. Testing The Temporal Properties of Black Box Systems Doron Peled, Bar Ilan University

2. Testing of black box finite state machine Wants to know: In what state we started? In what state we are? Transition relation Conformance Satisfaction of a temporal property Know: Transition relation Size or bound on size

3. Finite automata (Mealy machines) S - finite set of states. (size n) S– set of inputs. (size d) O– set of outputs, for each transition. (s0S - initial state). • S  S S - transition relation.  S  S O – output on edge.

4. Why deterministic machines? • Otherwise no amount of experiments would guarantee anything. • If dependent on some parameter (e.g., temperature), we can determinize, by taking parameter as additional input. • We still can model concurrent system. It means just that the transitions are deterministic. • All kinds of equivalences are unified into language equivalence. • Also: connected machine (otherwise we may never get to the completely separate parts).

5. a/1 b/1 a/1 Determinism When the black box is nondeterministic, we might never test some choices.

6. Preliminaries: separating sequences b/1 s1 s2 a/0 b/1 b/0 a/0 s3 a/0 Start with one block containing all states {s1, s2, s3}.

7. A: separate to blocks of states with different output. b/1 s1 s2 a/0 b/1 b/0 a/0 s3 a/0 Two sets, separated using the string b {s1, s3}, {s2}.

8. Repeat B: Separate blocks based on moving to different blocks. b/1 s1 s2 a/0 b/1 b/0 a/0 s3 a/0 Separate first block using b to three singleton blocks.Separating sequences: b, bb.Max rounds: n-1, sequences: n-1, length: n-1.For each pair of states there is a separating sequence.

9. State identification: • Want to know in which state the system has started (was reset). • Can be a preset distinguishing sequence (fixed), or a tree (adaptive). • May not exist (PSPACE complete to check if preset exists, polynomial for adaptive). • Best known algorithm: exponential length for preset,polynomial for adaptive [LY].

10. b/1 a/1 b/1 s1 s2 a/1 a/1 s3 b/0 Sometimes cannot identify initial state… thus need reliable reset Start with a:in case of being in s1or s3we’ll move to s1and cannot distinguish.Start with b:In case of being in s1or s2we’ll move to s2 and cannot distinguish. The kind of experiment we do affects what we can distinguish. Much like the Heisenberg principle in Physics.

11. Conformance testing • Unknown deterministic finite state system B. • Known: n states and alphabet . • An abstract model C of B. C satisfies all the properties we want from B. C has m states. • Check conformance of B and C. • Another version: only a bound n on the number of states l is known. 

12. b/1 a/1 b/1 s1 s2 ?= a/1 a/1 s3 b/0 Check conformance with a given state machine • Black box machine has no more states than specification machine (errors are mistakes in outputs, mistargeted edges). • Specification machine is reduced, connected, deterministic. • Machine resets reliably to a single initial state (or use homing sequence).

13. a/1 a/1 b/1 b/1 Conformance testing [Ch,V] a/1  b/1 b/1 a/1  a/1 b/1 Cannot distinguish if reduced or not.

14. a Conformance testing (cont.)  b b a a a  a b b a a b Need: bound on number of states of B.

15. b/1 a/1 b/1 s1 s1 s2 a/1 b/1 a/1 s3 s2 a/1 s3 b/0 Preparation:Construct a spanning tree

16. How the algorithm works? Reset According to the spanning tree, force a sequence of inputs to go to each state. • From each state, perform the distinguishing sequences. • From each state, make a single transition, check output, and use distinguishing sequences to check that in correct target state. Reset s1 a/1 b/1 s3 s2 Distinguishing sequences

17. Comments • Checking the different distinguishing sequences (n-1 of them) means each time resetting and returning to the state under experiment. • We assume a reliable reset. • Since, by assumption, there are no more than n states, and according to the experiment, no less than n states, there are n states exactly. • Isomorphism between the transition relation is found.

18. Combination lock automaton • Assume accepting states. • Accepts only words with a specific suffix (cdab in the example). c d a b s1 s2 s3 s4 s5 Any other input

19. When only a bound on size of black box is known… • Black box can “pretend” to behave as a specification automaton for n-l states, then upon using the right combination of size l, make a mistake. a/1 Pretends to be S1 for n -l states. b/1 a/1 b/1 b/1 s1 s2 a/1 a/1 s3 Pretends to be S3 b/0

20. Conformance testing algorithm [VC] Reset Reset • The worst that can happen is a combination lock branch that mimics usual behavior until the last state. The length of it is the difference between the bound n on the size of the black box and its actual size l. • Reach every state on the spanning tree and check every word of length n-l+1 or less. Check that after the combination we are at the state we are supposed to be, using the distinguishing sequences. • No need to check transitions: already included in above check. • Complexity: m2 n dn-l+1 Probabilistic complexity: Polynomial. s1 a/1 b/1 s3 s2 Words of length n-l+1 Distinguishing sequences

21. Model Checking • Finite state description of a system B. • LTL formula . Translate  into an automaton P. • Check whether L(B)  L(P)=. • If so, S satisfies . Otherwise, the intersection includes a counterexample. • Repeat for different properties.  

22. Buchi automata P (w-automata) S - finite set of states. (Black box B has l n states) S0S - initial states. (P has m states) S - finite alphabet. (contains p letters) d S  SS - transition relation. F S - accepting states. Accepting run: passes a state in F infinitely often. System automata: F=S, deterministic, one initial state. Property automaton: not necessarily deterministic.

23. Example: check a a <>a a a, a

24. a a a a Example: check <>a <>a

25. Example: check  <>a a, a <>a a a Use automatic translation algorithms, e.g., [Gerth,Peled,Vardi,Wolper 95]

26. a c b System

27. Every element in the product is a counter example for the checked property. a a <>a s1 s2 q1 a c b a a s3 q2 a s1,q1 s2,q1 Acceptance isdetermined byautomaton P. b a s1,q2 s3,q2 c

28. Given Finite state system B. Transition relation of B known. Property represent by automaton P. Check if L(B)  L(P)=. Graph theory or BDD techniques. Complexity: polynomial. Unknown Finite state system B. Alphabet and number of states of B or upper bound known. Specification given as an abstract system C. Check if B C. Complexity: polynomial if number states known. Exponential otherwise. Model Checking / Testing

29. Property represent by automaton P. Check if L(B)  L(P)=. Graph theory techniques. Unknown Finite state system B. Alphabet and Upper bound on Number of states of B known. Complexity: exponential. Black box checking [PVY]  

30. reset a a b b c c try c try b a a b b c c a a b c b c fail Experiments

31. Simpler problem: deadlock? • Nondeterministic algorithm:guess a path of length  n from the initial state to a deadlock state.Linear time, logarithmic space. • Deterministic algorithm:systematically try paths of length n, one after the other (and use reset), until deadlock is reached.Exponential time, linear space.

32. Deadlock complexity • Nondeterministic algorithm:Linear time, logarithmic space. • Deterministic algorithm:Exponential (p n-1) time, linear space. • Lower bound: Exponential time (usecombination lock automata). • How does this conform with what we know about complexity theory?

33. Modeling black box checking • Cannot model this problem using Turing machines: not all the information about B is given. Only certain experiments are allowed. • We learn the model as we make the experiments. • Can use the model of games of incomplete information.

34. Games of incomplete information • Two players: $-player, -player (here, deterministic). • Finitely many configurations C. Including:Initial Ci , Winning : W. • An equivalence relation @ on C (the $-player cannot distinguish between equivalent states). • Labels L on moves (try a, reset, success, fail). • The $-player has the same labels on moves from configurations that are equivalent (it does not distinguishes between them). • Deterministic strategy for the $-player: will lead to a configuration in W. Cannot distinguish between equivalent configurations. • Nondeterministic strategy: Can distinguish between equivalent configurations.

35. Modeling BBC as games • Each configuration contains an automaton and its current state (and more). • Moves of the $-player are labeled withtry a, reset... Moves of the -player withsuccess, fail. • c1@ c2 when the automata in c1and c2 would respond in the same way to the experiments so far.

36. A naive strategy for BBC • Learn first the structure of the black box. • Then apply the intersection. • Enumerate automata with n states (without repeating isomorphic automata). • For a current automata and newautomata, construct a distinguishing sequence. Only one of them survives. • Complexity: O((n+1)p (n+1)/n!)

37. On-the-fly strategy • Systematically (as in the deadlock case), find two sequences v1 and v2 of length <=m n. • Applying v1 to P brings us to a state t that is accepting. • Applying v2 to P brings us back to t. • Apply v1 v2 n-1 to B. If this succeeds,there is a cycle in the intersection labeled with v2, with t as the P (accepting) component. • Complexity: O(n2p2mnm). v1 v2

38. Learning an automaton • Use Angluin’s algorithm for learning an automaton. • The learning algorithm queries whether some strings are in the automaton B. • It can also conjecture an automaton Miand asks for a counterexample. • It then generates an automaton with more states Mi+1and so forth.

39. Angluin’s algorithm in a nutshell Distinguishing sequences(suffix closed) T 0 1 0 0 0 1 0 1 01 0 0 00 0 0 10 0 0 11 0 0 010 0 1 011 1 0 Data structures of table are related to spanning tree + distinguishing sequences in [VC]!! Access strings Z(prefix closed) Z.

40. Angluin’s algorithm in a nutshell Distinguishing sequences(suffix closed) T 0 1 0 0 0 1 0 1 01 0 0 00 0 0 10 0 0 11 0 0 010 0 1 011 1 0 A state: an equivalence class of rows with same values. Access strings Z(prefix closed) Z.

41. Angluin’s algorithm in a nutshell Distinguishing sequences(suffix closed) T 0 1 0 0 0 1 0 1 01 0 0 00 0 0 10 0 0 11 0 0 010 0 1 011 1 0 Closed: each row for Z. exists already for Z. In this example, this is not closed: no row with results like 011. Add row from Z.into Z. Update rows. Access strings Z(prefix closed) Z.

42. Angluin’s algorithm in a nutshell Distinguishing sequences(suffix closed) T 0 1 0 0 0 1 0 1 01 0 0 00 0 0 10 0 0 11 0 0 010 0 1 011 1 0 Consistent: successors of equivalent rows are equivalent. Since 001 we also expect 00010 This does not hold; add distinguishing sequence 01 to separate. Update columns. Access strings Z(prefix closed) Z.

43. A strategy based on learning • Start the learning algorithm. • Queries are just experiments to B. • For a conjectured automaton Mi , check if Mi  P =  • If so, we check conformance of Mi with B ([VC] algorithm). • If nonempty, it contains some v1 v2w . We test B with v1 v2n. If this succeeds: error, otherwise, this is a counterexample for Mi .

44. Complexity • m - size of the Buchi automaton. • l - actual size of B. • n - an upper bound of size of B. • d - size of alphabet. • Lower bound: reachability is similar to deadlock. • O(l 3 d l + l 2mn) if there is an error. • O(l 3 d l + l 2 n dn-l+1+ l 2mn) if there is no error. If n is not known, check while time allows. • Probabilistic complexity: polynomial.

45. Some experiments • Basic system written in SML (by Alex Groce, CMU). • Experiment with black box using Unix I/O. • Allows model-free model checking of C code with inter-process communication. • Compiling tested code in SML with BBC program as one process.

46. “Adaptive Model Checking”[Groce, P, Yannakakis, 2002] • What happens if we need to check a black box that is a variant of an old design? • Some changes are made, some parts remain. • Should we use “black box checking” from scratch? • Or can we use some elements that were already computed and tested?

47. “Adaptive model checking” • Instead of starting BBC from scratch, we can use from the old table: • Rows: these are the access strings of a spanning tree. • Columns: these are the separating sequences. • So: make the experiments first based on the rows and columns of the previous design, assuming that not so much was changed. • Rows don’t help when the system was prefixed by a password (i.e., a combination lock…).

48. “Grey box checking”[Elkind, Genest, P, Qu, 2006] • What happens when the system is a product AB, where A is known and B is black box? • Case 1: we can still experiment with B separately. Then it makes sense to do the learning separately on B, then do model checking. The model checking of the counterexample of the combined system, if exists, is projected on B for testing it.

49. “Grey Box Checking” • Case 2. We cannot test B separately. • Then consider a case where using VC algorithm will be exponential in the multiplication mn of sizes of A and B. • In this case, the naïve strategy (running all automata, and comparing pairs, each time removing one of them) has a lower complexity (exponential in pn log(n)).

50. Conclusions • Black Box Checking: automatic verification of unspecified system. • A hard problem, exponential in number of states, but polynomial on average. • Implemented, tested. • Another use: when the model is given, but is not exact. • (This has also been applied for compositional model checking.)