Object-Oriented Software Engineering Practical Software Development using UML and Java

1. Object-Oriented Software EngineeringPractical Software Development using UML and Java Chapter 10: Testing and Inspecting to Ensure High Quality Part 1: Basic Definitions Effective and Efficient Testing Lots of Personal Notes

2. Overview Testing and Inspections – two very important concepts / activities in software development. We often have ‘roles’ of testers; individual developers; for different ‘kinds’ of testing; independent test groups; quality assurance groups, etc. Quality Control; Editors; Various sizes and shapes…. Very unique to the environment within which the software is developed. We have many different types of tests too. Exist at many different levels for many different stakeholders. Some are automated; many are hands-on… Some tests test specific parts of system; others whole; Some at different stages of development; …. Many Many names!!!! Heuristic: a successful test is one that identifies a flaw. We will present a lot of very important material in these lectures. Chapter 10: Testing and Inspecting for High Quality

3. Tests – just a few types of tests…. White box testing; Black Box Testing Verification and validation (V&V) Coverage Testing: Branch testing, Path testing, Statement testing Alpha testing, Beta testing Acceptance Testing Unit test, Subsystem test, System test, Integrated system test, … Independent test groups Environmental System Testing; Operational Field Testing Running in parallel; Running live; Implement in pieces. Chapter 10: Testing and Inspecting for High Quality

4. Organizational Considerations Deficiency Report (DIREP) Burroughs Hardware / UNISYS Field Assistance Branch; Customer Service Tracked, prioritized, resources allocated... Reported to senior levels of management Incident Report – Honeywell Problem Ticket – Fix Tickets (LPS) Office of Primary Responsibility (OPR); Office of Collateral Responsibility… (OCR) Product Manager; Account Rep… Chapter 10: Testing and Inspecting for High Quality

5. 10.1 Basic definitions • A failure is an unacceptablebehavior exhibited by a system • The frequency of failures measures the reliability • Important design objective: achieve a • very low failure rate and hence • a high reliability. Chapter 10: Testing and Inspecting for High Quality

6. Basic definitions: Failure Causes: • A failure can result from a violation of an explicit or implicit requirement • Specific • functional requirement; • non-functional requirements… Chapter 10: Testing and Inspecting for High Quality

7. Basic definitions: MTBF MTTR Metrics • We clearly want to increase the Mean Time Between Failures (MTBF) and reduce the Mean Time to Repair (MTTR): • and damage caused by failures. • MTBF and MTTR (old engineering metrics): are considered measures of reliability. Chapter 10: Testing and Inspecting for High Quality

8. Basic Definitions - continued • A defect is a flaw in any aspect of the system that contributes, or may potentially contribute, to the occurrence of one or more failures • Might take several defects to cause a particular failure • Defects can occur anywhere – requirements, design, implementation, testing, etc. • They can also occur at any time! • True ‘War’ Story: “Mr. Smith said this couldn’t happen!” • An error is a slip-up or inappropriate decision by a software developer that leads to the introduction of a defect Chapter 10: Testing and Inspecting for High Quality

9. 10.2 Effective and Efficient Testing To testeffectively, you must use a strategy that uncovers as many defects as possible. Testing really attempts to find defects! Pure & simple! Chapter 10: Testing and Inspecting for High Quality

10. Effective and Efficient Testing • To test efficiently, find the • largest possible number of defects, using the • fewest possible tests. • Testing costs time, money and other resources! • Testing is not cheap! It is quite expensive! • This is where the problems come in. What is the expected return on investment? • There are many kinds of tests designed to uncover different kinds of defects. Chapter 10: Testing and Inspecting for High Quality

11. 10.2 Effective and Efficient Testing – more… Testing must be efficient. So what do we test?? We cannot test everything. (Exhaustive Testing is not possible – proven mathematically) Rather, we design different kinds of tests for ‘sufficient coverage.’ (we talk more about this in lecture 39) All kinds of tests: Many Categories Some are lumped into Coverage Tests. statement coverage branch coverage path coverage….. Others tests focus on creating appropriate test values (Equivalence Testing) Chapter 10: Testing and Inspecting for High Quality

12. Black-Box Testing Testers provide the system with inputs; observe outputs • They can see none of: • Source code • Internal data (no algorithms or data structures, …) • Design documentation describing the system’s internals • They simply do inputting based on requirements and observe outputs. • Testing Charge: Often depends on the kind and level of testing and what the ‘charge’ of the testing group (organization) really is. Chapter 10: Testing and Inspecting for High Quality

13. Black-Box Testing - more • Kind of testing end-users most frequently undertake – for obvious reasons. • Sometimes called • validation testing / • acceptance testing - IF done when system is declared ‘finished.’ • Black box testing may sometimes be called other things (alpha testing; beta testing…) but these tests (alpha and beta) usually have other connotations associated with them… • Black Box Testing is designed to show functionality and often (sometimes) satisfaction of non-functional requirements. (such as load considerations; simultaneously users, distribution, more) Chapter 10: Testing and Inspecting for High Quality

14. Verification and Validation (V&V) • Developers usually do their own testing first (verification) • Usually unit testing; testing within development team. • May involve subsystem testing, integrated testing. … • But, in general, the developers are testing their products. • Generally their testing is white box testing, but can be black box testing. • End users test system against requirements (validation) • This is essentially black box testing.

15. Glass-box testing – White-box testing Also called ‘structural’ testing Developers and Testers have access to the system design • They can • Examine the Design and these Documents • View the code for Standardization / conformance! • Observe at run time the steps taken by algorithms and their internal data • Individual programmers often informally employ glass-box testing to verify their own code and functionality • Developers (using tools) can tell what parts of their programs are executed the most (most time spent in them) and then these may be candidates for optimization…. Chapter 10: Testing and Inspecting for High Quality

16. Glass-box testing - White Box testing  Unit Testing Developers are always responsible for their own ‘modules’. modules (units) = programs, classes, subsystems, etc. Developers ‘verify’ their products. Developer programs implement the design Programs satisfy functional requirements, etc… A lot of work to undertake ‘unit testing.’ Hazardous though, since they may be required to simulate inputs, outputs, files passed, parameter values passed, etc…. Many kinds of unit testing… Chapter 10: Testing and Inspecting for High Quality

17. Glass-Box - White Box Testing - More Path Testing – exhaustive; impossible; there is an infinite number of paths in a non-trivial program. Branchtesting – Edge testing – the most feasible Design a test so that all of the edges of a node are executed. Often shown via flowgraphs. (ahead) What do we mean by ‘edge?’ (all ‘else’ branches taken; all ‘then’ branches executed…) Consider a flowgraph of an algorithm (two slides up) …. Emphasis will be on control and coverage. Chapter 10: Testing and Inspecting for High Quality

18. White box: Branch Testing - Flow Graphs Want minimum number of tests assuring highdegree of reliability. Know we cannot test all paths in a non-trivial program.  Want a ‘representative set.’ Given a flow graph (next slides), there are a couple of formulas that will provide the minimum number of tests . They are: • (number of nodes – number of edges + 1) • Number of Regions + 1 ….(cyclomatic complexity) Number of regions is considered to be a measure of complexity. Chapter 10: Testing and Inspecting for High Quality

19. http://www.scism.sbu.ac.uk/law/Section5/chap3/s5c3p22.html Consider: The number of possible paths through a subprogram is equal to the number of regions in the subprogram's flowgraph. The flowgraph below is for a SquareRoot function and has regions numbered, as shown. Can see there are four regions, as numbered, plus the outer region, or 5 in total. Chapter 10: Testing and Inspecting for High Quality

20. http://www.scism.sbu.ac.uk/law/Section5/chap3/s5c3p22.html This indicates that there are five possible paths through the flowgraph which can be described by listing the sequence in which the nodes must are visited. The five paths in the SquareRoot flowgraph are as follows. etc… Chapter 10: Testing and Inspecting for High Quality

21. White Box: Equivalence Class Testing • This is a Bigee! • It is inappropriate to test by brute force, using every possible input value • Discuss; Takes a huge amount of time • Is impractical and is pointless! • Far better: divide the possible inputs into groups which you believe will be treated similarly by all algorithms. • Such groups are called equivalence classes. • Remember Computational Structures?? (COT 3100??) Chapter 10: Testing and Inspecting for High Quality

22. Equivalence Class Testing • Recall ramdomizing in File Structures (hashing…). • If we used an ‘Equivalence mod 4’, we essentially divided the set of all non-negative integers by four and only take their ‘remainders’ 0, 1, 2, and 3. • If a numeric key was used as a primary or relative key, this approach was often used to arrive at a disk address or an address in a hash table. • Thus ‘Equivalence mod 4’ effectively partitions the set of all integers into one of five disjointequivalentclasses. • ALL divisions result in one of these remainders – and those having the same remainder (members of the same equivalence class) need only require one of its members to be tested to represent ALL values that fall into this equivalence class.. • We applied this equivalence relation to the set of integers and obtained a set of equivalence classes the Union of which constitutes the original set of Integers. • EC1 U EC2 U … ECn = set of I+ (non-negative) Chapter 10: Testing and Inspecting for High Quality

23. Example of equivalence classes • A tester need only to test one member of an equivalence class rather than ALL numbers of that class. • The tester has to • understand the required input, • appreciate how the software may have been designed • Example: • Valid integer input is a month number (1-12) • Equivalence classes are: [-∞..0], [1..12], [13.. ∞] • These are the three equivalence classes. • In Equivalence Class Testing, we select an input from each of the equivalence classes as inputs to testing. Chapter 10: Testing and Inspecting for High Quality

24. More on Equivalence Class Testing • So, to test for valid integers ranging from 1 through 12, we have three equivalence classes as indicated: • Integers less than 1; • Integers between 1 and 12 inclusive • Integers greater than 12. • Is this enough? • Equivalence Class Testing cannot handle all kinds of testing: • What if there is more than a single value we are testing? Chapter 10: Testing and Inspecting for High Quality

25. Combinations of Equivalence Classes • Combinatorial explosion means that you may not be able to realistically test every possible system-wide equivalence class. • Let’s say there are 4 inputs each of which have 5 possible values. This means that are 54 (i.e.625) possible system-wide equivalence classes. • While you should first make sure that at least one test is run with every equivalence class of every individual input, you should also test all combinations where one input is likely to affect the interpretation of another. Chapter 10: Testing and Inspecting for High Quality

26. Compatibility Tests; Associativity Tests • Compatibility tests; Associativity tests. • Example of Associativity Test: Example from a Manpower Project in the USAF. • Field/attribute: Rated Performance Indicator (rpi); • Rule: If the rpi input value for an individual had a value of non-zero ( individual had to be on an aircrew), then the ‘rank’ attribute for this individual (s/he must be an officer), must be in the range between O1-O10 (and could NOT be in the range E1-E9). • Example of Compatibility Test: • Range tests: If rank = ‘O4’, salary must fall between $nnnn and $nnnnnn. Many others and many different types of tests. Chapter 10: Testing and Inspecting for High Quality

27. Testing at Boundaries of Equivalence Classes Boundary Value Testing • More errors in software occur at the boundariesof equivalence classes than at any other place!!AMEN!!! • The idea of equivalence class testing should be expanded to specifically test values at the extremes of each equivalence class • E.g. The number 0 often causes problems •  E.g.: If the valid input is a month number (1-12) • Test equivalence classes as before • Test -24; 6, +24, (representative values) • Test boundaries: 0, 1, 12 and 13 as well as very large positive and negative values • Are only integers allowed? (perhaps another test) Chapter 10: Testing and Inspecting for High Quality

28. Coming Defects in ordinary algorithms…. There are many of these. Read carefully. Read / study slides for Lecture 38. Chapter 10: Testing and Inspecting for High Quality