CS 425/625 Software Engineering Software Testing

CS 425/625 Software EngineeringSoftware Testing Based on Chapter 20 of the textbook [Somm00] Ian Sommerville, Software Engineering, 6th Ed., Addison-Wesley, 2000 and on the Ch20 PowerPoint presentation available at the book’s web-site: www.comp.lancs.ac.uk/computing/resources/IanS/SE6/Slides/index.html November 12, 2002

Outline • Introduction • Defect Testing • Black Box Testing • Equivalence Partitioning • Structural (White Box) Testing • Path Testing • Integration Testing • Top-Down • Bottom-Up • Interface Testing • Stress Testing • Object Oriented Testing

Introduction.. • A very high level view of testing phases [Fig. 20.1, Somm00]

.Introduction. • Component testing is concerned with checking the functionality of methods, classes, and modules • Integration testing focuses on sub-systems and their interaction as well as on the operation of the system as a whole • Component testing is individual programmers’ task while integration testing is assigned to an integration team • For critical systems independent testing teams are used during all testing • Integration testing is based on written requirements specifications

..Introduction • In object-oriented testing the boundaries between component and integration testing are blurred because • Object-oriented systems do not have the clear distinction between program units and program modules characteristic to structured (function-oriented) systems • The hierarchy of objects is not necessarily “nested”

Defect Testing • Defect testing is aimed at discovering errors in code • A successful defect test causes the system to perform incorrectly • During the testing process the following are created: • Test cases • Test data • Test reports

Defect Testing • Comprehensive testing is not possible. Subsets of tests cases are designed based on criteria such as: • All code statements should be executed at least once • All functions accessed through menus should be tested • All combinations of functions accessed through the same menu should be checked • All user input situations should be checked with both correct and incorrect data

Defect Testing: The Process • The defect testing process [Fig. 20.2, Somm00]

Defect Testing: Black Box Testing. • In black box testing (or functional testing) the system is viewed from “outside,” only in terms of its functionality • The software is tested against its specification • Internal (implementation) details are ignored • The behavior of the system is evaluated in terms of inputs received, outputs expected, and outputs actually produced • A defect is discovered if the actual system output differs from the expected output • Applicable to both function-oriented and object-oriented systems

Defect Testing: .Black Box Testing • The black-box testing model [Fig. 20.3, Somm00]

Defect Testing: Equivalence Partitioning…. • Equivalence partitioning makes use of classes of input test data (e.g., positive numbers, negative numbers, strings, etc.) • Generally, systems tend to behave in a comparable way for all members of a class • Classes of input test data are called equivalence partitions or domains • Equivalence partitions are determined based on program specification and user documentation; • Test cases are derived for each equivalence partition • Partition mid-point and boundary values provide useful test data

.Defect Testing: .Equivalence Partitioning… • Equivalence partitioning [Fig. 20.4, Somm00]

..Defect Testing: Equivalence Partitioning.. • Equivalence partitions [Fig. 20.5, Somm00]

…Defect Testing: Equivalence Partitioning. • Example: A search routine [Fig. 20.6, Somm00] procedure Search (Key : ELEM ; T: ELEM_ARRAY; Found : in out BOOLEAN; L: in out ELEM_INDEX) ; Pre-condition -- the array has at least one element T’FIRST <= T’LAST Post-condition -- the element is found and is referenced by L ( Found and T (L) = Key) or -- the element is not in the array ( not Found and not (exists i, T’FIRST <= i <= T’LAST, T (i) = Key ))

….Defect Testing: Equivalence Partitioning • Equivalence partitions for search routine [Fig. 20.7, Somm00]

Defect Testing: Structural Testing…. • In contrast to black box testingstructural testing(orwhite box, clear box, glass box testing) is based on the knowledge the testers have about the structure and the implementation of the software • Applied generally to smaller program units • The tester analyzes the code to determine test cases • Types of structural testing: • Branch coverage: every branch of the unit is tested at least once • Statement coverage: every statement of the unit is executed at least once • Path coverage: every path through the unit is executed at least once

.Defect Testing: Structural Testing… • The structural testing process [Fig. 20.8, Somm00]

Structural Testing Example Java implementation [Fig. 20.9, Somm00]

Defect Testing: …Structural Testing. • Equivalence classes for binary search [Fig. 20.10, Somm00]

….Defect Testing: Structural Testing • Test cases for binary search [Fig. 20.11, Somm00]

Defect Testing: Path Testing… • Path testingis a form of structural testing aimed at exercising each individual execution path of a program unit. It ensures that • Each statement of the unit is executed at least once • Each branch condition is tested with both true and false conditions • The program flow graph is used for path testing. In a flow graph decisions are shown as nodes and the flow of control is represented by edges

Defect Testing: .Path Testing.. • The cyclomatic complexity CC of a connected graph G is given by the formula: CC(G) = Number of edges - Number of nodes + 2 • If there are no go to statements and no compound decisions (involving more than one test) then CC(G) = Number of predicate nodes + 1 (predicate nodes contain conditions that determine branching of the execution flow) • The cyclomatic complexity of a graph gives the minimum number of test cases needed to cover all paths

Defect Testing: ..Path Testing. • Flow graph for binary search [Fig. 20.12, Somm00]

Defect Testing: …Path Testing • Independent paths in previous case 1, 2, 3, 8, 9 1, 2, 3, 4, 6, 7, 2 1, 2, 3, 4, 5, 7, 2 1, 2, 3, 4, 6, 7, 2, 8, 9 • In programs with complex branching structure the number of paths is high and it is often difficult to predict the program’s behaviour • A dynamic program analyzer can be used to identify all paths

Integration Testing.….. • Integration involves building sub-systems from program units and the system from subsystems • Integration testing uses software specifications as basis for checking the system • Errors are more difficult to locate than in unit testing • The incremental approach is recommended for system integration and testing • Initially, a minimal system configuration can be used • Tests need be repeated after each addition to the system

.Integration Testing….. • Incremental integration testing [Fig. 20.13, Somm00]

..Integration Testing…. • Strategies for system integration and testing: • Top-Down • Starts with high level components and continues at lower levels • Integral part of the top-down development process • Stubs are needed to test the system

…Integration Testing… • Strategies for system integration and testing [cont’d]: • Bottom-Up • Starts with lower level components and builds up the system by adding & testing higher level components • Can be used when components from previously developed systems are reused or when the system’s functionality relies on critical lower level modules • Driver modules are used to test the system • “Sandwich” • Combines the above two strategies by using the Top-Down approach for logic modules (higher level) and the Bottom-Up approach for operational (lower level) modules

….Integration Testing.. • Top-down integration testing [Fig. 20.14, Somm00]

…..Integration Testing. • Bottom-up integration testing [Fig. 20.14, Somm00]

…...Integration Testing • Top-Down versus Bottom-Up: • Architectural validation • Top-Down allows earlier discovery of high-level design errors • System demonstration • Top-Down supports better an early demo; Bottom-Up can support such demo if many reusable components are utilized • Test implementation • Strict Top-Down testing is more difficult to implement given that stubs are needed. It is easier to use drivers in a strict Bottom-Up testing • Test observation • Equally challenging, since both stubs and drivers may not allow complete observations

Interface Testing… • Interface testing has the goal of detecting errors related to interfacing modules and sub-systems • Particularly important for object-oriented testing given objects’ collaboration relies on their interfaces • Types of interfaces: • Parameter interfaces • Shared memory interfaces • Procedural interfaces • Message passing interfaces • Classes of interface errors: • Interface misuse • Interface misunderstanding • Timing errors

.Interface Testing.. • Interface testing [Fig. 20.15, Somm00]: tests directed at the sub-system as a whole, not at particular components

..Interface Testing. • Guidelines for interface testing • Test all external calls with parameters at extreme ranges • When pointers are used, test interfaces with null pointers • For procedural interfaces try to design tests that would cause the components to fail • Use stress testing for message passing interfaces • Change activation order of components that share memory

…Interface Testing • Stress testing exercise the systems with overloading(e.g., number of transactions per minute, number of users in a distributed applications) • The main purposes of stress testing: • Make sure the system does not lose or corrupt data when overloaded • Reveal defects that otherwise would pass unnoticed

Object-Oriented Testing.. • Differences from function-oriented systems: • Objects are usually larger units, which encompass several methods • Objects are typically loosely coupled and there may not be an obvious high top of the class hierarchy • Testers may not have access to the code of reused objects

.Object-Oriented Testing. • Levels of testing in object-oriented testing: • Testing operations of object classes (sequences of operations may be needed) • Testing object classes (operations, attributes, and states) • Testing clusters of objects (use case or scenarios based testing; thread testing, etc.); each operation in each class should be checked at least once • Testing the object-oriented system

..Object-Oriented Testing • Compared with testing function-oriented systems, testing of object-oriented system is facilitated by inheritance + reuse (previously tested objects) but specific challenges exist because: • Information hiding is a complicating factor • Writing additional methods (invoked for testing purposes only) are often necessary • New and overwritten methods must be tested • Inherited methods must still be re-tested if they interact with newly written methods

CS 425/625 Software Engineering Software Testing