Testing

1. Testing Testing Techniques to Design Tests

2. Testing:Example Problem: Find a mode and its frequency given an ordered list (array) of with one or more integer elements. void getModeFreq (ArrayType a, int size, int & mode, int & freq); • Test case categories • Empty array ? • Normal cases? • More than one mode • All elements unique

3. Test to specification • Tests can have more than one answer and still be consistent with specification. • Find THE mode is a constraint on the values in the array. • Find A mode  several modes can exist. • Specification and test design developed concurrently

4. Black Box Testing • Happy Path • Interface Testing • Equivalence Partitioning • Boundary value analysis • Functional testing • Cause-effect graphing testing • Comparison testing • Error testing • Random inputs

5. Black Box Testing • Happy Path • Test cases and input values chosen from those known or expected to work. • Interface Testing • Unit interfaces and I/O interfaces (Automated test tools) • Equivalence Partitioning • Input class: generate 1 test point for each input class • Output class – generate 1 test point that gives result in each output class • Subdivide into subclasses (middle of range and 2 extremes) • Error Handling Routines/Exception output conditions (input beyond accepted range)

6. Boundary Value Analysis Derive test cases from boundaries of equivalence partitions. • Add equivalence partitions for output to input classes. • For each input and output class: • Write valid test cases for representative value and high and low values. • Write invalid test cases for values just beyond boundary.

7. Black Box Testing • Functional testing • Functions within certain math classes can be distinguished by their values on a small number of points F(x) = y, if x > 0; y-1, if x <= 0 • Functions of more than one variable? z=x+y+1 • Limitation: most programming problems not simple math problems • Random inputs • Cause-effect graphing testing • Comparison testing • Error testing

8. Error Guessing Test designer uses skill and experience to devise test cases to uncover errors. • null input. • long input. • random input. • almost correct input. • spaces in strings. • quoted strings. • all CAPS. • negative numbers.

9. White Box Testing • Based on program structure • Coverage metrics for thoroughness • Statement coverage : all statements are executed at least once. This is harder than it sounds: defensive programming makes some code hard to get to. if (f = fopen(path)) { perror("fopen"); } • Insufficient for “if P then S” (only need true condition to test statement)

10. White Box Testing • Branch coverage • all branches are taken • all conditions on branches are evaluated. • Includes statement coverage • Only this will catch bug above. • Consider • If (x < 0) • x--; • else • x++; • if (y < 0) • z = sqrt (-x); • else • z = x+y; Can branch cover with then-then and else-else without detecting the problem of calling sqrt with a negative value

11. White Box Testing • Path coverage (every combination of T/Fs) • Impractical – too many paths • Eliminate infeasible paths (no systematic way) If (y < 0) x = y –1; else x = y+1; if (x > 0) … then-then infeasible • Missing paths (“special cases”) – can involve 1 single input data point (if y = 0)

12. More…Path coverage • Coincidental correctness read x; y = x + 2; vs. y = x * 2; write y; • Can’t distinguish if input is 2 • Implies one point per path is insufficient

13. Paths 104 possible paths At 1 task/msec = 3170 years Loop <= 20

14. Data Flow Coverage All definitions and all uses of variables Mutation Analysis Create mutant programs See if mutant programs give identical output for each test; If yes, mutant is live If no, mutant is killed Display set of live mutants, if all killed, confidence in Test Examples: Replace 1 constant or variable with another Replace one arithmetic operator with another Similarly, relational and logical operators Delete a statement Increment change More White Box Testing