Software Testing Techniques

1. Software Testing Techniques “How to find bugs in the software?” TCS2411 Software Engineering

2. Lecture Objectives • To understand the role of testing in ensuring software quality • To discuss issues relevant to software testing • To describe the different techniques used for testing software TCS2411 Software Engineering

3. Validation & Verification • Validation • “Are we building the product right?” • Ensure software meets customer’s needs • Verification • “Are we building the right product?” • Ensure software meet specification (error-free) TCS2411 Software Engineering

4. Testing Terminology • Failure • Incorrect or unexpected output • Symptom of at fault • Fault • Invalid execution state • Symptom of an error • May or may not produce failure • Error • Defect or anomaly in source code • Commonly referred as bug • May or may not produce fault TCS2411 Software Engineering

5. What is Software Testing • Software Testing is a process of executing a program with the intent of finding an error. • A good test case is one that has a high probability of finding an as yet undiscovered error • A successful test is one that uncovers an as yet undiscovered error TCS2411 Software Engineering

6. Testing Objective • To design tests that systematically uncover different classes of errors and to do so with a minimum amount of time and effort TCS2411 Software Engineering

7. Why We Need Software Testing? • Reveal faults/failures/errors • Locate faults/failures/errors • Show system correctness • Improved confidence that system performs as specified (verification) • Improved confidence that system performs as desired (validation) • Indicator of system reliability and system quality TCS2411 Software Engineering

8. Importance of Testing • Critical Element of Software Quality Assurance • Well-planned and thorough testing are important in reducing the high cost of software failure • Can take up to 40% of development effort • In systems that require high reliability, testing can be 3 to 5 times of all the other steps combined TCS2411 Software Engineering

9. Testing Principles • All test should be traceable to customer requirements. • Tests should be planned before testing begins. • Testing should begin with individual components and move towards to integrated cluster of components. • The most effective testing should be conducted by an independent third party. TCS2411 Software Engineering

10. Pre-requisites For Good Testing • Adequate staff • Adequate support facilities (tools) • Adequate time TCS2411 Software Engineering

11. Who Should Test The Software? • Developer (individual units) • Independent test group (ITG) • removes conflict of interest • reports to SQA team TCS2411 Software Engineering

12. Successful Testing • Uncover errors in the software • Demonstrates that the software functions appear to be working according to specification • Data collected as testing is conducted provide a good indication of software reliability and software quality TCS2411 Software Engineering

13. Testing Guidelines • Testing a system’s capabilities is more important than testing its components • Testing old capabilities is more important than testing new capabilities • Testing typical situations is more important than testing boundary value cases TCS2411 Software Engineering

14. Test Cases • Test case : unit of testing activity • Test cases have 3 parts :- • Goal • Aspect of the system being tested • Input and system state • Data provided to the system under stated condition • Expected behavior • The output or action the system should take according to its requirements TCS2411 Software Engineering

15. time money resources people Test Cases (Cont) • Test cases are derived during all phase of development cycle • Determine the expected result before running the test case • Record test cases and its results • Allocate TCS2411 Software Engineering

16. Type Of Test Cases • Test cases are derived for • Valid and expected input • Invalid and unexpected input • Test if the system does less than specified requirement • Test if the system does more than specified requirement TCS2411 Software Engineering

17. White Box Testing • Derived from knowledge of program’s structure & implementation • Structural testing - analyse code & use knowledge of the structure of a component to derive test data • Logical paths are tested by providing test cases that exercise specific sets of conditions and/or loops TCS2411 Software Engineering

18. White Box Testing (Continued) • Thorough white box testing would lead to “100 percent correct programs” • Exhaustive testing are impractical - too many tests! • A limited number of logical paths can be selected and exercised • Important data structures can be probed for validity TCS2411 Software Engineering

19. White Box Test Cases • Guarantee that all independent paths have been exercised at least once • Exercise all logical decisions on their true and false sides • Execute all loops at their boundaries and within their operational bounds • Exercise internal data structures to ensure their validity TCS2411 Software Engineering

20. White Box Testing Techniques • Basis path testing • Flow graph notation • Cyclomatic complexity • Derived test cases • Graph metrics • Control structure testing • Condition testing • Data Flow testing • Loop testing TCS2411 Software Engineering

21. 1 Edge Node 1 2,3 2 6 4,5 R2 3 7 8 R1 R3 6 4 Region 9 7 8 5 9 10 10 11 R4 11 Flow Graph Notation TCS2411 Software Engineering

22. Cyclomatic Complexity • Provide quantitative measure for program logical complexity. • Defined number of independent path • Any path that introduce one ser of processing statements or new condition • Eg :- • Path 1 : 1-11 • Path 2 : 1-2-3-4-5-10-1-11 • Path 3 : 1-2-3-6-8-9-10-1-11 • Path 4 : 1-2-3-6-7-9-10-1-11 TCS2411 Software Engineering

23. How Is Cyclomatic Complexity Computed? • Number of regions • The flow graph has 4 regions • V(G) = E – N + 2 • E : Number of flow graph edges • N : Number of flow graph nodes • V(G) = 11 edges – 9 nodes + 2 = 4 • V(G) = P + 1 • P : Number of predicate nodes • V(G) = 3 predicate nodes + 1 = 4 TCS2411 Software Engineering

24. … i=1; total.input = total.valid=0; sum=0; do while value[i] <> -999 and total.input<100 increment total.input by 1; if value[i] >= minimum AND value[i] <= maximum then increment total.valid by 1; sum = sum + value[i] else skip end if increment i by 1 End do If total.valid > 0 then average = sum / total valid; else average = -999; End if … 2 1 3 4 6 5 7 8 9 10 11 12 13 Deriving Test Cases • Draw flow graph from design/code as foundation. 1 2 3 4 10 12 11 5 13 6 8 7 9 TCS2411 Software Engineering

25. Deriving Test Cases (cont) • Determine cyclomatic complexity • V(G) = 6 regions • V(G) = 17 edges – 13 nodes + 2 = 6 • V(G) = 5 predicates nodes + 1 = 6 • Determine a basis set of linearly independent graph • Path 1 = 1-2-10-11-13 • Path 2 = 1-2-10-12-13 • …. • Prepare test cases • Path 1 test case : • value(k) = valid input where k < i defined below • value(i) = -999 where 2 <= i <= 100 • Expected result : Correct average based on k value and proper totals TCS2411 Software Engineering

26. Discussion on White Box Testing • Advantages • Find errors on code level • Typically based on a very systematic approach, covering the complete internal module structure • Constraints • Does not find missing or additional functionality • Does not really check the interface • Difficult for large and complex module TCS2411 Software Engineering

27. Black Box Testing • Derived from program specification • Functional testing of a component of a system • Examine behaviour through inputs & the corresponding outputs • Input is properly accepted, output is correctly produced • Disregard internal logical structure TCS2411 Software Engineering

28. Black Box Testing (Continued) Attempts to find the following errors: • Incorrect or missing functions • Interface errors • Errors in data structures or external database access • Performance errors • Initialisation and termination errors TCS2411 Software Engineering

29. Black Box Testing Techniques • Graph Based Testing Methods • Equivalence Partitioning • Boundary Value Analysis • Comparison Testing • Orthogonal Array Testing TCS2411 Software Engineering

30. Equivalence Partitioning • Divide input domain into classes of data • Based on an evaluation of equivalence classes for an input condition • Guidelines to define equivalence classes • Range input : One valid and two invalid equivalence • Specific value : One valid and two invalid equivalence • A member of a set : One valid and one invalid equivalence • Boolean : One valid and one invalid equivalence TCS2411 Software Engineering

31. Example – Data for Automated Banking Application The use can access the bank using his personal computer, provide a six digit password, and follow with a series of typed commands that trigger various banking function. During the log on sequence the software supplied for the banking application accepts data in the form: area code – blank or 3 digit numbers prefix – 3 digit numbers, nor beginning with 0 or 1 suffix – 4 digit numbers password – 6 digits alphanumeric strings commands – “check”, “deposit”, “bill pay” etc Input condition area code : Input condition : Boolean – area code may or may not present Input condition : Range – 200 – 999 with specific exception prefix : Input condition : Range – specified value > 200 with no 0 digits suffix : Input condition : Value – 4 digit length password : Input condition : Boolean – password may or may not present Input condition : Value – six character string command : Input condition : Set – containing command TCS2411 Software Engineering

32. Boundary Value Analysis • Complement equivalence partitioning • Test both sides of each boundary • Look at output boundaries for test cases • Test min, min-1, max, max+1, typical values • Example : 1 <= x <=100 • Valid : 1, 2, 99, 100 • Invalid : 0 and 101 TCS2411 Software Engineering

33. Discussion on Black Box Testing • Advantages • Find missing functionality • Independent from code size and functionality • Find some coding errors • Constraints • No systematic search for low level errors • Specification errors not found TCS2411 Software Engineering

34. References • “Software Engineering: A Practitioner’s Approach” 5th Ed. by Roger S. Pressman, Mc-Graw-Hill, 2001 • “Software Engineering” by Ian Sommerville, Addison-Wesley, 2001 TCS2411 Software Engineering