Data Flow Testing

1. Data Flow Testing Csci 565 Spring 2009

2. Objectives • Define/Use Testing • DU-Path Test Coverage Metrics • Example Commission problem

3. Test coverage criteria • The main focus is structural testing • Data flow • Control flow • Mutation testing • How much testing is enough? • need to define test coverage criteria (or yard stick) • Test coverage criteria? • A minimal set of test requirements (e.g. p0, p1, etc.) that must be collectively exercised by the test cases when we execute the program

4. The testing process of Data Flow

5. Test Coverage criteria • Control flow analysis • Uses flow graph • Control flow Coverage Criteria • All-nodes • All-edges • All-paths

6. Informal Definitions of The DF Testing • Refers to the form of structural testing (or path testing) • used for compiler optimization • Attempts to “bridge the gap” between branch testing and path testing • Suffers from un-executable paths (infeasible paths) • The focus is on the locations (i.e., statements) where variables are defined and used • A definition (value is stored in MM) • A Use (a value is fetched from MM) • Data Flow diagrams (or charts) are modeling notations used at different abstraction levels

7. Data flow analysis (DFA)’s anomalies • Detects Defines/reference anomalies, such as • A variable that is defined but never used • (unused variable) • A variable that is used but never defined • (undefined variable) • A variable that is defined twice before it is used • (redundant operations) • Data reads from location not previously written to • (uninitialized variables) • The anomalies can be discovered by “static analysis”

8. Define/Use Testing • Early work was done in 1980s by [Rapps 85] • The research adopts the formalization of program graph in which • Nodes correspond to program statements • Edges correspond to the flow of information • Single entry and single exist points • No self-looping nodes

9. Defining node? • Node n G(P) is a defining node of the variable v  V, written as DEF(v, n), • iff the value of the variable v is defined at the statement fragment corresponding to node n • G(P) refers to Flow Graph of Program P

10. Example of defining statements • Defining Statements include • Input statements (e.g., read) • Assignments statements • Loop control statements • Procedure calls • When the code corresponding to the above statements executes, the contents of the memory locations related to the variables are changed

11. Usage node? • Node n G(P) is a usage node of the variable v  V, written as USE(v, n), • iff the value of the variable v is used at the statement fragment corresponding to node n

12. Example of Usage statements • Usage Statements include • Output statements (e.g., write) • Assignments statements (r.h.s.) • Conditional statements • Loop control statements • Procedure calls • When the code corresponding to the above statements executes, the contents of the memory locations related to the variables are unchanged

13. Predicate use: (P-use) • A usage node USE(v, n) is either • a predicate use (denoted by P-use) • Or, a computation use (denoted by C-use) • The nodes corresponding to P-use have out-degree  2 • The nodes corresponding to C-use always have outdegree 1

14. Definition-use (sub)path • A definition-use (sub)path w.r.t. a variable v (denoted du-path) is a (sub)path in PATHS(P) such that for some vV there are defined and usage nodes • DEF(v, m) and USE(v,n) • Where • m is initial node • n is final nodes

15. Definition-clear (or dc) path • A defintion-clear(sub)path w.r.t. a variable v (denoted dc-path) is a DU(sub)path in PATHS(P) with initial and final nodes DEF(v, m) and USE (v, n) such that no other node in the (sub)path is a defining node of v • DF testing • Requires that the test data execute dc-path from each node containing a global definition of a variable to specified nodes containing global c-uses and edge containing p-uses of that variable

16. The significant of DU-Paths and DC-Paths • Du-paths and dc-paths capture the essence of computing with stored data values • Du-Paths and dc-Paths describe the flow of data across source statements from the locations where the values are defined to the locations where the values are used • Du-paths that are not Dc-paths are considered as problematic spots

17. About VAR and CONST • VAR and CONST declaration • Should they be considered as defining nodes? • Yes and no

18. Example: The Commission problem discussed in [Jorgenson95] • The Commission Problem Statement • Rifle salesperson in the Arizona Territory sold rifle locks, stocks, and barrels made by a gunsmith in Missouri. Locks cost $45.00, stocks cost $30.00, and barrels cost $25.00. Salespersons had to sell at least one complete rifle per month, and production limits are such that the most one salesperson could sell in one month is 70 locks, 80 stocks, 90 barrels. Each rifle salesperson sent a telegram to the Missouri company with the total order for each town s/he visits; salespersons visit at least on town per month, but travel difficulties made ten towns the upper limit. At the end of each month, the company computed commission as follows: • 10% on sales up to $1000 • 15% on the next $800 • 20% on any sales in excess of $1800. • The company had four salespersons. The telegram from each salesperson were sorted into piles (by person) and at the end of each month a data file is prepared, containing the salesperson's name, followed by one line for each telegram order, showing the number of locks, stocks, and barrels in that order. At the end of the sales data lines, there is an entry of “-1” in the position where the number of locks would be to signal the end of input for that salesperson. The program produces a monthly sales report that gives the salesperson’s name, the total number of locks, stocks, and barrels sold, the salesperson's total dollar sales, and finally his/her commission.

24. Some Du-paths • Stocks variable • DEF(stocks, 25) • USE(stocks, 27) • DuPath • P0=<25, 27> • Dc-Path • <25,27> • Locks variable • DEF(locks,22) • DEF(locks, 29) • USE(locks,23) • Use(locks,26) • Du-path • p1=<22,23> • p2=<22,23,24,25,26> (begin the loop) • p3<29,30,23> • p4=<29,30,23,24,25,26> (repeat the loop) • p1’=<22,23,31> (by pass the loop) • p3’=<29,30,23,31> (exist the loop) • Dc-paths • p1,p2,p3,p4,p1’,p3’ • Complete set of test cases for the WHILE-loop: p1’, p2, p3’, p4

26. DU-paths w.r.t. num_locks • DU-Paths w.r.t. num_locks • Used in computational uses (c-uses) • Defining nodes • DEF(num_locks,19) • DEF(num_locks,26) • Usage nodes • USE(num_locks,26) • USE(num_locks,33) • USE(num_locks,36) • DU-paths • P5=<19,20,21,22,23,24,25,26> (dc-path) • P6=<19,20,21,22,23,24,25,26, 26, 27,28,29,30,31,32,33> (NOT dc-path) • Compacted p6: • P6=<19,20,21,22,23,24,25,26, 27,28,29,30,31,32,33> • p7 = <19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36) • p7 can also be rewritten as P7= <p6,34,35,36> (NOT dc-path because it contains node 26) • P8=<26,27,28,29,30,31,32,33> (subpath of p7; dc-path) • P9=<26,27,28,29,30,31,32,33,34,35,36> (subpath of p7; dc-path)

27. DU-paths w.r.t. sales • Du-paths w.r.t. sales • ONLY one defining node for sales (i.e. the paths are dc-paths) • p10=<36,37> • p11=<36,37,38,39> • p12=<36,37,38,39,40,41,42,43> (dc-use) • Look at IF, ELSE IF (statements 39-50) • p13=<36,37,38,39,45,46,47,48> • p14=<36,37,38,39,45,50>

28. DU-paths w.r.t. Commission • Du-paths w.r.t. Commission • <43,44,45,48,50,51> (no cp-path) • Consider only du-paths that begin with three “real” defining nodes for commission • p15 =<43,51> • p16=<48,51> • p17=<50,51> • Table 4 shows the full set of du-paths

30. DU-PATH TEST Coverage Metrics (or Rapps-Weyuker dataflow criteria) • Idea behind analyzing a program is to define a set of test coverage (AKA the Rapps-Weyuker DF criteria) • ALL-paths • All-edges • All-nodes • All-Defs • All-Uses • All P-Uses/Some C-Uses • ALL-C-Uses/Some P-Uses • All-DU-paths

31. ALL-Defs • The set T Satisfies the All-Defs criterion for the program P iff • for every variable vV, T contains dc-paths from every defining node of v to a use of v where • V: a set of variables • T: a set of (sub)paths in G(P) of a program P

32. ALL-Uses • The set T Satisfies the All-Uses criterion for the program P iff • for every variable vV, T contains dc-paths from every defining node of v to every use of v , and to the successor node of each USE(v,n) • where • V: a set of variables • T: a set of (sub)paths in G(P) of a program P

33. ALL-P-Uses/Some C-Uses • The set T Satisfies the All-P-Uses/Some C-Uses criterion for the program P iff • for every variable vV, T contains dc (sub)paths from every defining node of v to every predicate use of v , and if a definition of v has no P-uses, there is a definition-clear path to at least one computation use • where • V: a set of variables • T: a set of (sub)paths in G(P) of a program P

34. ALL-C-Uses/Some P-Uses • The set T Satisfies the All-P-Uses/Some C-Uses criterion for the program P iff for every variable vV, T contains definition clear (sub)paths from every defining node of v to every use of v , and if a definition of v has no C-uses, there is a definition-clear path to at least one predicate use • where • V: a set of variables • T: a set of (sub)paths in G(P) of a program P

35. All-DU Paths • The set T Satisfies the All-DU paths criterion for the program P iff for every variable vV, T contains definition clear (sub)paths from every defining node of v to every use of v, and to the successor node of each USE(v,n), and that these paths are either single loop traversals, or they are cycle free

38. Summary of the relevant terms

39. Summary: Du-paths

40. Summary of the test coverage criteria

41. Simple Example:1

42. Simple Example:2

43. References • [Rapps 85] S. Rapps and E. Weyuker. Selecting software test data using data flow information. IEEE Transaction on Software Eng., vol.se-11, no.4, pp:367-375, April, 1985. • [ Frankl 88] P. Frankl and E. Weyuker. An Applicable Family of Data Flow Testing Criteria. IEE Transaction on software eng., vol.14, no.10, October 1988. • [Weyuker 88] E. Weyuker. The evaluation of Program-based software test data adequacy criteria. Communication of the ACM, vol.31, no.6, June 1988. • [Jorgenson 02]. Software Testing: A Craftsman’s Approach.2nd CRC publication, 2002.