Testing

1. Testing CS 123/CS 231

2. References • Debugging and Testing • Reiss, Chapter 8 • Verification and Validation • Sommerville, Chapters 22-24 • Testing Techniques and Strategies • Pressman, Chapters 17, 18, 23

3. Debugging & Testing: Definitions • Bug • a defect or error in a program • Testing • running a program with the intent of finding bugs (defect testing) • Debugging • determining the exact nature of a bug

4. Testing Principles • All tests should be traceable to customer requirements • Tests should be planned long before testing begins • Pareto principle (80-20) applies to testing

5. Testing Principles, continued • Begin testing “in the small” and progress towards testing “in the large” • Exhaustive testing is not possible • Most effective testing is done by third parties

6. Types of Bugs • Syntactic Bugs • syntax errors, typos • Design Bugs • Logic Bugs • Interface Bugs • method invocation versus definition • Memory Bugs

7. Steps in Debugging • Locating the problem/cause • the more difficult step (in general) • use inductive/deductive reasoning • Repairing the problem • if design bug, consequences are often extensive • note that solution may result in more bugs • regression testing

8. Defensive Design • Simplicity • in class interfaces • in algorithms • Encapsulation • minimizes coupling; increased correlation between locations of error and cause • Error-handling • design with error in mind

9. Defensive Coding • Writing code to simplify debugging • What to watch out for: • initialization • method preconditions (check for valid parameter values) and return value cases • complex code / bug “history” • Language support • assertions (in C,C++ - assert.h), exceptions

10. Reviews • Particularly for large projects with several designers and programmers involved • Design Review • presentation to other developers, stakeholders, and outside consultants • brings out other, external perspectives • Code Review • walk through your code, line by line

11. Testing • May be performed at different levels • Static Testing • testing without program execution • code review, inspection, walkthrough • Dynamic Testing • module testing, integration testing • system/regression testing • black-box vs white-box testing

12. Stages in the Testing Process • Unit Testing • Module Testing • Sub-System Testing • System/Integration (Validation) Testing • Acceptance Testing * “in the small” -> “in the large”

13. How to Test • Devise Test Plan • clear/consistent procedure/standards for testing a system • Identify Test Cases • thorough listing of possible inputs • include boundaries, invalid/unexpected inputs • Implement the Test • write test code (test drivers), use testing tools

14. Test Cases • Equivalence Partitioning • partition input (& output) into classes that are processed equivalently by the unit • Test Case Guidelines • Example: for arrays, consider varying sizes, test for first, middle, and last element • Path Testing • based on possible execution paths; white-box

15. (OO) Test Case Design • In the small • Class-level (Unit testing) • Generate (random) operation sequences on an object • In the large (cluster based testing, use-case based testing) • Objective: test class collaboration • Integration testing

16. Test Case Example(Unit Testing) • Example: BankAcount class • Some Random Operation Sequences • create.deposit.deposit.balance.withdraw. withdraw.close • create.withdraw.deposit.balance.withdraw.close • Enumerate input cases for each class method • ensure that operation sequences cover all the input/parameter cases • withdraw: balance >= 0, amt < balance, …

17. Integration Testing • Include collaborating classes • Example: BankAccount, ATMGui, and Customer • Formulate operations according to use case. e.g., • encode-customer-info, create-account, setup-pin, withdraw, deposit, get-balance.