1 / 53

Black-Box Testing Techniques III

Black-Box Testing Techniques III. Software Testing and Verification Lecture 6. Prepared by Stephen M. Thebaut, Ph.D. University of Florida. Another Cause-Effect Example: Symbol Table Storage Specification. The conditions for storing an identifier in one of two symbol tables are:

rodd
Télécharger la présentation

Black-Box Testing Techniques III

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Black-Box Testing Techniques III Software Testing and Verification Lecture 6 Prepared by Stephen M. Thebaut, Ph.D. University of Florida

  2. Another Cause-Effect Example: Symbol Table Storage Specification The conditions for storing an identifier in one of two symbol tables are: (a) must be from 2 to 8 characters in length; (b) first character must be a letter or “$”; (c) other characters must be a letter or digit. If the first character is a letter, the identifier will be stored in symbol table A. If the first character is “$”, it will be stored in symbol table B. If the first character is neither a letter nor “$”, or if condition (c) is not satisfied, error message J11 is output. If condition (a) is not satisfied, error message J12 is output.

  3. Causes and Effects Causes:Effects: (1) 2  no. chars  8 (31) store in table A (2) 1stchar is letter (32) store in table B (3) 1stchar is $ (33) output msg J11 (4) other charsonly letters/digits (34) output msg J12 only (35) output msgs J11 and J12

  4. Boolean Graphs [2,8] let $ (1) (2) (3) (4) (31) (32) −> A −> B E others let/dig

  5. Boolean Graphs (cont’d) [2,8] let $ (1) (2) (3) (4) Л (31) (32) −> A −> B E others let/dig

  6. Boolean Graphs (cont’d) [2,8] let $ (1) (2) (3) (4) Л (31) (32) −> A −> B E Л others let/dig

  7. Boolean Graphs (cont’d) [2,8] let $ (1) (2) (3) (4) (33) (34) (35) J11 only J12 only J11 & J12 E others let/dig

  8. Boolean Graphs (cont’d) [2,8] let $ (1) (2) (3) (4) Л (33) (34) (35) J11 only J12 only J11 & J12 Л E (A) V (B) others let/dig

  9. Boolean Graphs (cont’d) [2,8] let $ (1) (2) (3) (4) Л (33) (34) (35) J11 only J12 only J11 & J12 Л Л E (A) V (B) others let/dig

  10. Boolean Graphs (cont’d) [2,8] let $ (1) (2) (3) (4) Л (33) (34) (35) J11 only J12 only J11 & J12 Л Л E (A) V (B) Л others let/dig

  11. A Variation on Test Case Selection Strategy #3 • Test case selection “Strategy #3” (p. 32) considers ALLfeasible combinations of connected Cause values that result in each Effect being True. • For complex specifications, this can be impractical. • We now consider a variation on this strategy which “culls” all but the combinations “of greatest interest”.

  12. Test Case Selection Strategy #3 Plus “Culling Rules” REPEAT Select the next (initially, the first) Effect. Tracing back through the graph (right to left), find all feasible combinations of connected Cause values that result in the Effect being True, subject to the following culling rules: • When encountering an nth-degree OR-node that must be True, consideronly those n combinations for which exactly one incoming edge is True.

  13. Test Case Selection Strategy #3 Plus “Culling Rules” (cont’d) • When encountering an nth-degree AND-node that must be False, consider only those n combinations for which exactly one incoming edge is False. For each new such combination found: Determine values of all other Effects, and Enter values for each Cause and Effect in a new column of the test case coverage matrix. UNTIL each Effect has been selected.

  14. Rationale for these Culling Rules? • Number of combinations decreases by a factor of O(2) to O(n) at each true OR node and each false AND node. • Idea: cover only the minimally sufficient conditions for the desired result. n ( ( V Л T F

  15. Applying Strategy #3 Plus Culling Rules [2,8] let $ (1) (2) (3) (4) Л (31) (32) −> A −> B E Л others let/dig

  16. Coverage Matrix

  17. Applying Strategy #3 Plus Culling Rules (cont’d) [2,8] let $ (1) (2) (3) (4) Л (31) (32) −> A −> B E Л others let/dig

  18. Coverage Matrix (cont’d)

  19. Applying Strategy #3 Plus Culling Rules (cont’d) [2,8] let $ (1) (2) (3) (4) Л (33) (34) (35) J11 only J12 only J11 & J12 Л Л E (A) V (B) Л others let/dig

  20. Applying Strategy #3 Plus Culling Rules (cont’d) (33)  1, B B  (A V 4)  (4, A) V (4, A) V (4, A) (T,T) (T,F) (F,T) culled (rule 1)

  21. Applying Strategy #3 Plus Culling Rules (cont’d) (33)  1,4, A 1, 4, A

  22. Applying Strategy #3 Plus Culling Rules (cont’d) (33) 1,4, A 1, 4, A A  2,3

  23. Applying Strategy #3 Plus Culling Rules (cont’d) (33) 1,4, A 1, 4, 2,3

  24. Applying Strategy #3 Plus Culling Rules (cont’d) (33) 1,4, A A  (2, 3) 1, 4, 2,3

  25. Applying Strategy #3 Plus Culling Rules (cont’d) (33)  1,4, A A  (2, 3)  (2, 3) V (2, 3) V (2, 3) (F,F) (F,T) (T,F) culled (rule 2) and infeasible 1, 4, 2,3

  26. Applying Strategy #3 Plus Culling Rules (cont’d) (33) 1, 4, 2, 3 1, 4, 2, 3 1, 4, 2,3

  27. Coverage Matrix (cont’d)

  28. Applying Strategy #3 Plus Culling Rules (cont’d) [2,8] let $ (1) (2) (3) (4) Л (33) (34) (35) J11 only J12 only J11 & J12 Л Л E (A) V (B) Л others let/dig

  29. Applying Strategy #3 Plus Culling Rules (cont’d) (34)  1, B B (4 V A)  4, A

  30. Applying Strategy #3 Plus Culling Rules (cont’d) (34)  1, 4, A

  31. Applying Strategy #3 Plus Culling Rules (cont’d) (34)  1, 4, A A  (2, 3)

  32. Applying Strategy #3 Plus Culling Rules (cont’d) (34)  1, 4, A A  (2, 3)  (2, 3) V (2, 3) V (2, 3) (F,F) (F,T) (T,F) culled (rule 2) and infeasible

  33. Applying Strategy #3 Plus Culling Rules (cont’d) (34)  1, 4, 2, 3  1, 4, 2, 3

  34. Coverage Matrix (cont’d)

  35. Applying Strategy #3 Plus Culling Rules (cont’d) [2,8] let $ (1) (2) (3) (4) Л (33) (34) (35) J11 only J12 only J11 & J12 Л Л E (A) V (B) Л others let/dig

  36. Applying Strategy #3 Plus Culling Rules (cont’d) (35) 1, B Which are just the conditions associated with error messages J11 (B) and J12 (1). Combining these conditions from (33) and (34) yields: (35) 1, 4, 2, 3 1, 4, 2, 3 1, 4, 2,3

  37. Coverage Matrix (cont’d)

  38. Complete Coverage Matrix

  39. Cause-Effect Analysis: Discussion Questions & Exercises • Under what circumstances should Cause-Effect Analysis be used for test case design? • Are there any other obvious benefits? • What are the pros and cons of having a set of mutually exclusive Effects?

  40. Cause-Effect Analysis: Discussion Questions & Exercises (cont’d) • Suppose that some program Effect is associated with integer input X being either 30 or even. What are the pros and cons of defining { X | X  30 V EVEN(X) } to be a Cause. • Devise a scenario that illustrates some creative ideas for how a well-engineered CASE tool could effectively support Cause-Effect Analysis.

  41. Cause-Effect Analysis: Discussion Questions & Exercises (cont’d) • Cause-Effect Analysis seems well suited for “single-state-transition” program models in which Causes are mapped to Effects in one conceptual step. How could you apply the strategy to multi-state-transition program models?

  42. Boundary Value Analysis • A technique based on identifying, and generating test cases to explore boundary conditions. • Boundary conditions are an extremely rich source of errors. • Natural language based specifications of boundaries are often ambiguous, as in “for input values of X between 0 and 40,...”

  43. Boundary Value Analysis (cont’d) • May be applied to both input and output conditions. • Also applicable to white box testing (as will be illustrated later).

  44. Guidelines for Identifying Boundary Values • “Range” guideline: K will range in value from 0.0 to 4.0. • Identify values at the endpoints of the range and just beyond. • Boundary values: 0.0-(I) 0.0 (V) 4.0 (V) 4.0+(I)

  45. Guidelines for Identifying Boundary Values (cont’d) • “Number of values” guideline: The file will contain 1-25 records. • Identify the minimum, the maximum, and values just below the minimum and above the maximum. • Boundary values: empty file (I), file with 1 (V), 25 (V), and 26 (I) records

  46. Boundary Value Analysis Exercise Identify appropriate boundary values for the following program specification fragment.

  47. City Tax Specification 1: The first input is a yes/no response to the question “Do you reside within the city?” The second input is gross pay for the year in question. A non-resident will pay 1% of the gross pay in city tax. Residents pay on the following scale: - If gross pay is no more than $30,000, the tax is 1%. - If gross pay is more than $30,000, but no more than $50,000, the tax is 5%. - If gross pay is more than $50,000, the tax is 15%.

  48. Test Case Design Based on Intuition and Experience • Also known as Error Guessing, Ad Hoc Testing, Artistic Testing, etc. • Testers utilize intuition and experience to identify potential errors and design test cases to reveal them. • Guidelines: • Design tests for reasonable but incorrect assumptions that may have been made by developers.

  49. Intuition and Experience (cont’d) • Guidelines: (cont’d) • Design tests to detect errors in handling special situations or cases. • Design tests to explore unexpected or unusual program use or environmental scenarios.

  50. Intuition and Experience (cont’d) • Examples of data conditions to explore: (1) (2) Repeated instances or occurrences (3) Repeated instances or occurrences (4) Bl anks or null char acters in strings (eT c.) (-5) Negative numbers (#) Non-numeric values in numeric fields (or vic3 versa) (6789) Inputs that are too long or too short

More Related