Automated Software Testing

11. Automated Software Testing

12. Contents Introduction to Automated Software Testing Process of Automated Testing Advantages & Disadvantages Choosing Automation Tools

13. Introduction to automated software testing “Software Testing is usually the part of the project which makes grown men weep” [1] Ever-shrinking schedule and minimal resources It involves automating a manual process of testing

14. Definition of Automation “Automation is the use of tools and strategies that reduce human involvement or interaction in unskilled, repetitive or redundant tasks” [2]

15. THE AUTOMATED TEST LIFECYCLE METHODOLOGY [4]

16. Why automate? Verification and Validation of the requirements and organisation Defects and Failures caused by misunderstood requirements or coding errors Compatibility of the Software

17. Advantages & Disadvantages of Automated Testing Reliable: eliminating human error Reusable Better Quality Software Fast Cost Reduction High investment is needed in the tools and training High man power requirement for test preparations Alot of testing areas left uncovered Disadvantages Advantages

18. Choosing Automation Tools • Ease of integration • Compatibility • Performance • Types of tests • Maintainability • Affordability

19. OBJECT ORIENTED TESTING SYSTEM TESTING INHERITANCE INTEGRATION TESTING POLYMORPHISM ENCAPSULATION UNIT TESTING By Maj Nicko Petchiny

20. REFERENCES • Developing an OO Software Testing and Maintenance Environment (King, Gao, Hsia, et-al) • Incremental Testing of OO Class Structures (Harrold, McGregor) • OO Integration Testing (Jorgensen, Ericksen) • OO Software Testing, A Hierarchical Approach (Siegel)

21. OUTLINE • TRADITIONAL VS OO SW DEVELOPMENT AND TESTING • OO CONCEPTS/EFFECT ON TESTING • PROPOSED OO INTEGRATION TESTING APPROACH • EXAMPLE USING TEST APPROACH • CONCLUSION

22. TRADITIONAL DEVELOPMENT & TESTING (WATERFALL LIFE CYCLE) • REQUIREMENTS SPEC SYSTEM TESTING • PRELIMINARY DESIGN INTEGRATION TESTING • FUNCTIONAL • DECOMPOSITION • DETAILED DESIGN UNIT TESTING

23. TRADITIONAL TESTING • SYSTEM • VERIFY SW SATISFIES ALL SW REQRS • INTEGRATION • BASED ON STRUCTURE OF DESIGN • TOP DOWN OR BOTTOM UP APPROACH • UNIT • ENCAPSULATES FUNCTIONALITY

24. OO DEVELOPMENT & TESTING • DEVELOPMENT BASED ON BEHAVIOUR • COMPOSITION • TYPICALLY RAPID PROTOTYPING • INCREMENTAL APPROACH • 3 TRADITIONAL TESTING LEVELS ARE NOT AS CLEARLY DEFINED

25. OBJECT ORIENTED TESTING • SYSTEM • SAME AS TRADITIONAL • STILL BASED ON REQRS SPEC • UNIT • TWO COMMON STRUCTURES USED • METHOD* • CLASS • SAME AS TRADITIONAL(DRIVERS & STUBS)

26. F OBJECT CLASS A METHOD 1 METHOD METHOD 2 E METHOD B C D METHOD METHOD METHOD

27. F OBJECT CLASS A METHOD 1 METHOD METHOD 2 E METHOD D B C METHOD METHOD METHOD

28. OO INTEGRATION TESTING • MAIN PROGRAM IS MINIMIZED • MOST COMPLICATED PART OF OO TESTING • TESTING BASED ON COMPOSITION IN BOTTOM UP APPROACH • USE OF CLUSTERS • ORD - CLASS DEPENDENCIES • BBD OR DIRECTED GRAPHS - SHOWS METHOD DEPENDENCIES

29. A OUTPUT PORT EVENT Class 3 Class 1 A meth2 meth1 1 meth1 meth2 meth3 B B OUTPUT PORT EVENT Class 2 2 meth1 meth3 3 MM-Path meth2 Message

30. OO CONCEPTS/EFFECTS ON TESTING • ENCAPSULATION • POLYMORPHISM • INHERITANCE

31. ENCAPSULATION • CLASS STRUCTURE • INTERFACE DEFINED BY PUBLIC METHODS • BEHAVIOR DEFINED BY METHODS THAT OPERATE ON ITS INSTANCE DATA (IN CONVENTIONAL SEPARATE) • HELPS ENFORCE INFO HIDING

32. ENCAPSULATION TESTING ISSUES • MINIMIZES RIPPLE EFFECT (AT THE UNIT LEVEL) OF MAKING A CHANGE • HIGHLY DELOCALIZED • CHANGE COULD RESULT IN SIGNIFICANT REGRESSION TESTING • ORDER OF TESTING IS IMPORTANT (CAN REDUCE TESTING EFFORT)

33. CLASS A CLASS C METHOD METHOD USES USES CLASS B METHOD

34. POLYMORPHISM • AN ATTRIBUTE MAY HAVE MORE THAN ONE SET OF VALUES • AN OPERATION MAY BE IMPLEMENTED BY MORE THAN ONE METHOD ( e.g GRAPHICS ) • OVERLOADING (type or number of variables) • DYNAMIC BINDING

35. OO TESTING ISSUES • POLYMORPHISM • DO YOU TEST ONE VARIANT ? • DO YOU TEST ALL VARIATIONS ? • IF ALL, DO YOU TEST ALL VARIANTS AT ALL LEVELS • UNIT • “INTEGRATION” OR SYSTEM LEVEL • REUSE DRIVERS AND STUBS

36. INHERITANCE STRUCTURES SINGLE MULTIPLE MULTIPLE LEVELS BASE BASE BASE BASE SUBCLASS SUBCLASS SUBCLASS SUBCLASS

37. INHERITANCE RESULT CLASS PARENT CLASS + MODIFIER

38. INHERITANCE B C A A B + A M1 B + C M1 B C + + M2 M2

39. INHERITANCE MODIFIERS • NONE (ONLY INHERITED ATTRIBUTE) • ADD NEW ATTRIBUTE(S) • REDEFINE PARENT’S ATTRIBUTE(S) • VIRTUAL ATTRIBUTE (THREADS IN JAVA)

40. OO TESTING ISSUES • INHERITANCE • DO YOU COMPLETELY TEST ALL BASE CLASSES AND THEIR SUB-CLASSES ? • DO YOU COMPLETELY TEST ALL BASE CLASSES AND ONLY TEST THE CHANGES OR MODIFICATIONS IN THEIR SUB-CLASSES ? • AT WHAT LEVELS DO YOU TEST? • IN WHICH ORDER DO YOU TEST?

41. INHERITED TESTING

42. OO TESTING METHODOLOGY • JORGENSEN AND ERICKSEN PROPOSE 5 LEVELS • A METHOD - UNIT TESTING • MESSAGE QUIESCENCE - INTEGRATION • EVENT QUIESCENCE - INTEGRATION • THREAD TESTING -SYSTEM • THREAD INTERACTION -SYSTEM

43. CONSTRUCT DEFINITIONS • MM-PATH (METHOD MESSAGE - PATH) [MESSAGE QUIESCENCE] • SEQUENCE OF EXECUTIONS LINKED BY MESSAGES. • STARTS WITH METHOD AND ENDS WITH A METHOD THAT DOESN’T PRODUCE A MESSAGE

44. CONSTRUCT DEFINITIONS • ASF (ATOMIC SYSTEM FUNCTION) [EVENT QUIESCENCE] • REPRESENTS AN INPUT EVENT • FOLLOWED BY A SET OF MM-PATHS • TERMINATED BY AN OUPUT EVENT