A Test Case + Mock Class Generator for Coding Against Interfaces

1. A Test Case + Mock Class Generator for Coding Against Interfaces Mainul Islam, Christoph Csallner Software Engineering Research Center (SERC) Computer Science and Engineering Department University of Texas at Arlington, USA February 07, 2011. • Published at: 8th International Workshop on Dynamic Analysis (WODA), 2010

2. Talk Outline • Test case generation • Dynamic Symbolic Execution • Mock Class • Motivating examples to generate Mock Class • Algorithm to generate Mock Class • Results • Demonstration Mock Class Generator for Coding Against Interfaces

3. Test Case Generation • What is the goal behind testing ? public void Test(int x) { if (x > 0) { • if (x == 1234567890) { • throw new Exception(“Bug”); • } } } Probability to reach the bug (with a random test input) is: ½ * 1/2^31 • What can be a good test case to test this method ? Mock Class Generator for Coding Against Interfaces

4. Symbolic Execution • Explore all feasible execution paths • Initialize the input with symbolic values and execute the program over symbolic values • At conditional statements check if either of the branches can be taken (using a constraint solver) • For each path we get an accumulated path condition If (s) then … else … C s true false C` = C ⋀ s C` = C ⋀⌝s • Dynamic symbolic execution: Combine Concrete and Symbolic execution Mock Class Generator for Coding Against Interfaces

5. Dynamic Symbolic Execution by Example Choose Next Path void TestMe (int a[]) { if (a == null) return; if (a.length> 0) if (a[0] == 123) throw new Exception(“Error”); } Solve Execute Constraints to Solve Input Observed Constraints a == null a == null null a != null {} a != null && !(a.length> 0) a == null T F a != null && a.length> 0 {0} a != null && a.length> 0 && a[0] != 123 a.length> 0 {123} T a != null && a.length> 0 && a[0] == 123 a != null && a.length> 0 && a[0] == 123 F a[0] == 123 T F Done: No path left This example is taken from the slides of Nikolai Tillman (with permission)

6. Introduction to Mock Class public interface A { // … } public class C { public void foo (Aa, int x) { // … // … } } Mock Class Generator for Coding Against Interfaces

7. When we might need to generate mock-classes ? • At the initial stage of any development Mock Class Generator for Coding Against Interfaces

8. Why is this important ? • Test case generation is important • Interfaces are very important • Existing techniques are not very good at this Pex - can generate an instance of a class that implements an interface that is used in the code * * http://research.microsoft.com/en-us/projects/pex/ Mock Class Generator for Coding Against Interfaces

9. Goal: • Support interfaces in test case generation for object-oriented programs Mock Class Generator for Coding Against Interfaces

10. Motivating Example 1 public interface A { int m1(int x); } • public interface B { • int m2(int x); • } public class C { public void foo (Aa, int x) { int y = a.m1(x); if (a instanceofB) { Bb= (B) a; int z = b.m2(x); } } } To reach this block of code, ‘a’ must be an instance of A, as well as an instance of B Mock Class Generator for Coding Against Interfaces

11. Motivating Example 2 public interface A { int m1 (int x); } public interface B { int m2 (int x); } public class K implements A, B { // … } public class C { public void bar (Aa, int x) { if (a == null) return; if ( !(a instanceofB) ) { Bb= (B) a; int z = b.m2(x); } } } K implements all of the interfaces referred by the bar method, but fails reach the code block Mock Class Generator for Coding Against Interfaces

12. Framework • DSC – Dynamic Symbolic Execution Engine (for Java) • Z3 – Constraint Solver http://research.microsoft.com/en-us/um/redmod/projects/z3/ Mock Class Generator for Coding Against Interfaces

13. Our approach to generate Mock Class • Determining the type of the Mock Class • Generate meaningful method body if needed Mock Class Generator for Coding Against Interfaces

14. Sub-/Supertype relation in Java • Java defines a binary subtype relation • If type B implements/extends type A then, - A is a direct super type of B - B is a direct sub type of A Reflexive:A is also a subtype of itself Transitive: if B is a subtype of A and C is a subtype of B then C is also a subtype of A Mock Class Generator for Coding Against Interfaces

15. Sub-/Supertype relation in Java (Cont.) • A class has one direct class super type and arbitrarily many interface super types. • Exceptions: type object – has no direct super type type null – has no direct sub type Mock Class Generator for Coding Against Interfaces

16. Subtype constraints Object Object public interface A { int m1 (int x); } public interface B { int m2(int x); } public class C { public void foo (Aa, int x) { int y = a.m1(x); if (a instanceofB) { Bb= (B) a; int z = b.m2(x); } } } A B C A B C M null Initial types in the system null A desired solution: with new type M Constraints: type(a) subtypeofA type(a) != null type type(a) subtypeofB Mock Class Generator for Coding Against Interfaces

17. Subtype Relation Matrix public interface A { int m1 (int x); } public interface B { int m2(int x); } public class C { public void foo (Aa, int x) { int y = a.m1(x); if (a instanceofB) { Bb= (B) a; int z = b.m2(x); } } } Solution: MA = true MB = true MC = false Mock Class Generator for Coding Against Interfaces

18. Algorithm: Generating Mock Class Program execution Initialization: NO_MC = 0 MAX_MC = 2 Build Subtype Relation matrix (Generate the constraints) Yes Found Solution? No Solution not found NO_MC < MAX_MC No Yes Yes Introduce a new Mock Class, N0_MC++ Encode the Mock Class in the type system Generate Test Case Dynamic Symbolic Data Structure Repair

19. Generating Method Body: for Example 1 public interface A { int m1 (int x); } public interface B { int m2(int x); } public class C { public void foo (Aa, int x) { int y = a.m1(x); if (ainstanceofB) { Bb= (B) a; int z = b.m2(x); • if (z >= 10) { • //… • } } } } • public class M implements A, B { • int m1 (int x){ • return 0; • } • int m2(int x){ • return 0; • } } Outline of the source code for Mock Class M return 10; Mock Class Generator for Coding Against Interfaces

20. Results • We have run our solution on JMS (Java messaging system) • These are all hand-written test cases Mock Class Generator for Coding Against Interfaces

21. Comparison with Pex: Motivating Example 1 public interface A { int m1(int x); } • public interface B { • int m2(int x); • } public class C { public void foo (Aa, int x) { int y = a.m1(x); if (a instanceofB) { Bb= (B) a; int z = b.m2(x); } } } Goal 1 Goal 2 PEX reaches 1/2 goal(s) DSC reaches 2/2 goal(s) Mock Class Generator for Coding Against Interfaces

22. Future Work • Extend the solution to support more types, e.g. ‘Array’ • Determine a upper bound to the number of mock classes needed to solve a system • Evaluation Mock Class Generator for Coding Against Interfaces

23. Future Work (Cont.) • Extend method body generation for complex types public class C { public void foo (Aa, int x) { int y = a.m1(x); if (ainstanceofB) { Bb= (B) a; int z = b.m2(x); • if (z >= 10) { • //… • } } } } • public class M implements A, B { • int m1 (int x){ • return 0; • } • int m2(int x){ • return 0; • } } Outline of the source code for Mock Class M return 10; Mock Class Generator for Coding Against Interfaces

24. Demo Mock Class Generator for Coding Against Interfaces

25. Thank you! Mock Class Generator for Coding Against Interfaces