CSIS3600 Systems Analysis and Design

1. CSIS3600 Systems Analysis and Design System Implementation and Testing

2. Testing • Once coding has begun, the testing process can also begin. • Testing is an important part of development. • Initial testing plans are usually devised during the analysis stage to be carried out during the implementation phase. • Besides testing the software for errors and defects, testing serves as a way to improve communications. • Analyst serves as an advisor and coordinator for program testing.

3. Quality Assurance and Testing • QA activities are often shortchanged during design and especially during implementation.

4. Schedule pressures can build with each successive phase. • QA and testing activities may be bypassed in an ill-fated attempt to speed up the project.

5. QA activities require development personnel to open their work to thorough examination and criticism by others. • Many people are reluctant to do this.

6. Many people view testing and test personnel as the bearers of bad news. • They mistakenly believe that no news is good news and that bypassing testing is a way of avoiding bad news.

7. How to Overcome the Attitude Towards QA • Integrate QA into the project and schedule it from the beginning and never abandon it.

8. What Must Be Tested? • Code (application programs, etc.) • Procedural manuals • New Hardware • System Interfaces

9. When should testing really begin? • Ideas?

10. When Should Testing be Done? • Done throughout the systems development process • Best to test beforehand than to fail after installation • Testing can be done on 'parts' and then in totality

11. Categories of Tests • Testing can be divided several ways. • Generally there are two categories of tests. • Static testing in which the code being tested is not executed • Technical Reviews, Walkthroughs, Inspections and Desk checking are static • Dynamic testing in which the software code is executed.

12. Formal but less Intimidating Quality Assurance Procedures • Technical review • Formal or informal review of design or construction details by a group of developers • Sometimes it just helps for someone else to ‘look at it’

13. Walkthrough • A review by two or more people of accuracy and completeness. • Tend to be informal • The focus is on what the code does • Most often used during analysis and design but can also be used during implementation • If done during analysis, done without actual code (can be done using models) • [somewhat what I do when I review your group deliverables]

14. DeskChecking • Testing technique in which the program code is sequentially executed manually by the reviewer • Usually someone who understands program logic reviews the code with a pencil and paper

15. Inspection • More formal version of a walkthrough • Participants review and analyze materials before they meet as a group • When group meets, participants play specific roles • Presenter – summarized material being inspected • Critics – describe errors or concerns found • Secretary – records notes • Additional errors or problems may be uncovered during the discussion • Usually the goal is to examine program code for predictable language-specific errors • Participants discuss possible solution strategies and, before the meeting is over, agree on approaches to problem resolution.

16. Formal QA Procedures • Testing usually follows four phases: • Modeling the software's environment • Selecting test scenarios • Running and evaluating test scenarios • Measuring testing progress

17. Test Cases • A test case is a formal description of • a starting state • one or more events to which the software must respond • the expected response or ending state

18. Starting state and ending state are represented by a set of test data. • Starting state may represent a particular set of database contents • existence of a particular customer and order for that customer • Event may be represented by a set of input data • customer account number and order number used to query order status • Expected response may be a described behavior (display of certain information) or a specific state of stored data (canceled order)

19. More on Test Cases • Preparing test cases and test data is tedious and time-consuming. • At the program or module level, every instruction must be executed at least once. • There are automated tools based on proven mathematical techniques that can be used to generate a complete set of test cases.

20. What Tests to Select • Set of tests that cause each source statement to be executed at least once • Set of tests that cause each branching structure (If, CASE, While, etc) to be evaluated with each of its possible actions. • Set of tests that cause each data structure to be initialized and then subsequently used • Set of tests that expose the seeded faults. • Seeded faults are errors intentionally inserted into the source code. Ideally by finding seeded errors, the tester will also find real errors.

21. Tests • Unit Testing • Integration Testing • Stub Testing • System Testing • Build and Smoke Test • Regression Testing

22. Unit Testing • Testing of each module in isolation in an attempt to discover any errors in its code. • Programmers are generally responsible for unit-testing their own code prior to integrating it with modules written by other programmers. • Some organizations assign “testing buddies” to help programmers test their own code. • Having a different programmer test the code usually finds more errors.

23. Integration Testing • Process of bringing together modules for test purposes • Objective is to identify errors that were not or could not be detected by unit-testing such as: • Interface compatibility • Parameter values • Run-time exception • Unexpected state interactions • It is a gradual process, • You being with the root and add coordinating modules one at a time

24. Stub Testing • Process used in testing modules • Stubs are 2-3 lines of code that stand in for a missing module • Effective where modules operate top-down • During testing the coordinating module calls a stub who ‘stands’ in for the missing module.

25. System Testing • Similar to integration testing but extends the bringing together of program modules to bringing together components. • Follows the same incremental logic as integration testing. • More than simply expanding integration testing • attempts to demonstrate whether the system has met its objective • Involves the use of non-live test data in a non-live testing environment that simulates the real environment as closely as possible. • Usually test case scenarios are developed to form the basis of system testing

26. Build and Smoke Test • System test that is performed daily • The system is completely compiled and linked and a battery of tests is performed to see whether anything malfunctions in an obvious way (“smokes”) • Commonly associated with interactive or rapid development • Identify problems results from code modification • Daily testing ensure that errors are found quickly and can be easily tracked to their source

27. Regression Testing • After testers submit successfully reproduced failures to development, developers generally create a new version of the software (in which the bug[s] have been removed). • Testing progresses through subsequent software versions until one is determined to be fit for release. • This retesting is called regression testing.

28. Acceptance Test • Performed to determine whether the system fulfills user requirements • Last round of testing before system is handed over to users • Formal activity

29. Versioning • Alpha • Beta • Limited Availability • General Availability

30. Alpha • Test version that is incomplete but ready for some level of rigorous testing. • Tests performed during alpha include: • Recovery testing – force software to fail to insure it can recover • Security testing – verify protection mechanisms • Stress testing – tries to break the system • Performance testing – determines how the system performs under a range of possible environments.

31. Beta • A system that is stable enough to be tested by end users • Run in an end users environment using their own data • Sometimes thought of as a rehearsal of the installation phase

32. Production version • Limited Availability – A production version considered final but is released to a limited number of end users for additional user acceptance testing • General Availability – A production version is considered a final product that can be formally distributed to general population of users

33. Numbering of Versions • Alpha usually starts at 0.# • Beta is also numbered beginning with 0.# • First Production version starts with 1.0 • Versions progress from here sometimes with alpha and beta versions of major upgrades

34. Date January 2000 Alpha version 0.1 April 2000 Alpha version 0.2 July 2000 Beta version 0.3 September 2000 Beta version 0.4 January 2001 Production version 1.0 April 2001 Planned version 2.0 September 2001 Production version 1.1 January 2002 Alpha version 1.91. April 2002 Beta version 1.9.2 January 2003 Production version 2.0 Example Timeline – Test and Production

35. Complexity of Testing OO • Methods can be called by many other methods and the calling methods may be distributed across many classes • Classes may inherit methods and state variables from other classes • The specific method to be called is dynamically determined at run time based on the number and type of message parameters • Objects can retain internal variable values (that is, the object state) between calls. • The response to two identical calls may be different due to state changes that result from the first call or occur between calls.

36. Testing Summation • Testing and Technical reviews find between 50 and 75 percent of errors. • Some errors are more easily detected using one method while other by using a different method. • Thus, the two techniques are more effective used jointly rather than individually.

37. References: • Types of Testing Coverage: • http://www.kaner.com/coverage.htm • Step-by-Step Test Design • http://www.stqemagazine.com/ • Number of testers to developers: • http://www.jrothman.com/Papers/ItDepends.html • Testing and Java Technology • http://developer.java.sun.com/developer/technicalArticles/InnerWorkings/testing/ • Site for Testers to Share Techniquies • http://www.testingcraft.com/exploratory-robinson.html • Great Bugs Site • http://www.bugnet.com/