Testing Levels for Object-Oriented Software

1. Advanced Software Testing Testing Levels forObject-Oriented Software

2. Introduction • Testing approaches must be revisited for OO software • This paper addresses determination of testing levels • Testing levels correspond to subsets of the target program to be tested in a certain order • The traditional unit and integration levels defined for procedural programs do not fit well in the case of OO programs • Functions must be considered together • They act on a shared state • Smallest unit to be tested is a class

3. … continued • Complex dependency between classes • The determination of testing levels may depend on the number of stubs to be constructed

4. Challenges • Complex relationships between classes • Dynamics (execution-time) relationships between classes, due to polymorphism • Abstract classes (cannot be instantiated) • Minimizing the number of stubs to be constructed

5. Kung et al’s Test Order • Based on Object Relation Diagram • Edge-labeled graph • Nodes represent classes • Edges represent inheritance, aggregation, and association relations • ORD captures the static dependencies between classes.

6. An example ORD

7. … continued • Construct ORD by reverse engineering C++ programs • Class firewall • Test order finding algorithm

8. Class Firewall • CFW(X): The set of classes that could be affected by changes to class X and thus, should be retested when class X is changed • Assume that the changes does not affect the ORD • CFW(X) has to include • Child classes of X • Aggregated classes of X • Associated classes of X • CFW(X) = {Ck | there is a directed path from Ck to X in the ORD} • S = {X1, …, Xq}, CFW(S) = CFW(X1)  CFW(X2)  … CFW(Xq)

9. Example

10. Test Order for Class Firewalls • Desirable test order • One that requires minimum effort to construct test stubs • Test the independent classes first, and then test the dependent classes based on their relationships • Algorithm: In case of an acyclic ORD, perform topological sorting • X is tested before all the classes of CFW(X), stubs are not required

11. Example • Changes: classes A and D • CFW(A, D) = {B, C, E, F, G, H}, so all the classes must be retested.

12. … continued • For a cyclic ORD: • Transform the cyclic ORD into an acyclic digraph ORD’ where the nodes represent clusters (maximal set of nodes that are mutually reachable through the relation R), apply topological sorting to produce test orders for clusters • For each cluster, break the cycle and use topological sorting to produce a test order for the classes in the cluster • To break a cycle, (temporarily) remove one association edge

13. Comments • Limitations of Kung et al’s approach: • Dynamic relationships between classes are not taken into account (due to polymorphism) • The test order provides information on the classes to be tested at each level. But, it does not give explicit information on the other classes that have to be involved in the test experiments • Some test levels become (partly) infeasible in cases of abstract classes so that the test order should be revised

14. The New Approach - Overview • We assume that cycles in the ORD have already been broken • Steps • Analyze both static and dynamic dependencies between the classes • Define the testing levels from the results of the analysis • Define a partial ordering relation to produce a test order • Define test criteria to be associated with each testing level, by indicating the role of each class involved in the level • Remove infeasible testing levels due to abstract classes from the graph

15. Class Dependencies Analysis • D1(X) is the set of classes on which X depends statically • D2(X) is the set of classes on which X depends either statically or dynamically (at execution time) or both • Boolean function Bd(X) indicates whether or not X may dynamically depend on at least one class of D2(X), due to polymorphism

16. … continued • A class X statically depends on a class Ck if and only if there is a directed path from X to Ck in the ORD • If Cj is a server class of class X then, at execution time, X may depend on all the (direct or by transitivity) child classes of Cj • C-ORD: ORD + (dotted) edges that indicate dynamic dependencies

17. Example A class X depends on a class Ck either statically or dynamically or both if and only if there is a directed path from X to Ck in the C­ORD Although we consider only acyclic ORDs, dynamic dependencies can introduce cycle(s) in the C­ORD, that is, we may have X in D2(X). In that case, all the classes in the cycle have the same set D2(X)

18. Boolean Function Bd(X) • Class X dynamically depends on other class(es) if and only if at least one class Ck in D2(X)  {X} has a dotted outgoing edge in the C­ORD • Bd(X) = 1, if and only if X dynamically depends on other class(es)

19. Example

20. Testing Levels • A testing level T is described by a triplet (T.goal, T.need, T.type) • T.goal is the set of classes under test (CUT) • T.need is the set of classes that have to be involved in the test experiment; it contains the classes under test (T.goal) and all the classes on which they are dependent according to T.type; • T.type indicates the type of dependencies taken into account: either static (Sta) or dynamic (Dyn)

21. … continued • D1(X) gives the minimum set of classes required to test class X, (at compile time) • D2(X) gives the maximum set of classes required to test class X, (when taking polymorphic relationships into account) • Bd(X) indicates whether or not a testing level with T.type = Dyn is to be defined

22. … continued • A testing level T = ({X}, {X}  D1(X), Sta) is associated with each class X • To capture dynamic dependences, the following testing levels are defined: • ({X}, {X}  D2(X), Dyn), in case there is no cycle involving X • ({X1, ..., Xj}, D2(X1), Dyn) otherwise, where X1, ..., Xj are all the classes involved in a same cycle

23. Partial Ordering of Testing Levels • Let M = (M.goal, M.need, M.type) and N = (N.goal, N.need, N.type) be two testing levels. We say that level M precedes level N if and only if • M.type = N.type and M.need  N.need, or • (M.type =Sta and N.type = Dyn) and M.need  N.need • The ordering is represented by a graph G = [V, E], the vertices (V) are the testing levels, the edges (E) indicate immediate precedence. • The graph shows which testing levels must be treated in sequence and which ones may be treated independently.

24. Example

25. Adding Information to Testing Levels • Re­introduce some information existing in the C-ORD in order to • facilitate the identification of infeasible testing levels • later assignment of test criteria to the remaining levels • A textual representation of testing levels • As a list of class names with additional special characters denoting the role of the classes within the testing level

26. … continued • Writing conventions: • Name of each class under test (belongs to T.goal) is decorated with the special character '#'. • A class whose only role in T.need is to be a parent class must not be instantiated. Its name is written between parentheses • A class may be both a parent and a server class • If T.type = Dyn, this class introduces polymorphism, which is indicated by the special character '*‘ • If T.type = Sta, these relations are tested only through instances of the (parent) server class according to static dependency. This case is indicated by the special character '+‘

27. Example ({C}, {C,D}, Sta)  C#,(D) ({F}, {C,D,F,G,H}, Dyn)  (C),(D),F#,G*,H

28. Accounting for Abstract Classes • For each abstract class, the test order graph is modified in the following two ways: • When its role implies its instantiation, the level is infeasible and has to be removed. This is done by deleting the corresponding vertex and merging its incoming and outgoing edges. The test of CUTs in the level is postponed to the next feasible testing levels (special character '#' is added) • The class may be both a parent and a server class ('+' and '*'). In this case, the abstract class loses its role of server class. (The server relationship will only be instantiated with children of the abstract class). The only role of the class is to be a parent class: parentheses are added to special characters '+' and '*'

29. Example • Assume A and D are abstract • Removed: • A# • A,B#,C,(D) • D# • Changed: • C#,(D)  C#,(D)# • A+,B,C,(D),E#  (A+)#,B#,C,(D),E# • A*,B#,C*,(D),E#,F,G*,H  (A*),B#,C*,(D), E#,F,G*,H

30. Example

31. TOONS Tool • Input • the list of the static relationships between the classes • information about the presence of abstract classes • Performs dependency analysis, defines testing levels, orders them, and produces the test order graph using the writing conventions

32. Future Work • Possible modifications of the graph to incorporate additional information on the classes • Account for the semantics of the classes under test • Simple classes as a mere driver • Reuse of classes developed in the framework of other applications • First step toward the definition of an incremental strategy for testing OO programs • Precise criteria focusing on the coverage of either intra­class or inter­class features, or both

33. … continued • The most challenging issue: Testing of highly integrated subsystems involving many polymorphic server classes