TPM Software Testing IntroductionTPM – Test Process Model Marcio Cunha Test Team Lead
Objectives • Understand why The Test Process Model • Identify reasons for testing • Describe key testing concepts • Describe the testing life cycle
Models and Process ISO IEE CMMi mpsBR SEI CMM Quality Metrics Processes
O Bom Senso é a coisa do mundo melhor compartilhada entre os homens, pois cada um pensa estar tão bem provido dele, que mesmo os que nunca se contentam com o que têm, não desejam ter ainda mais Bom senso do que já o têm. Common Sense and Good Sense Rene Descartes
Is it Built right with acceptable quality ? Customer Developer The user requirements are built as specified ... has 'the system been built right‘ ? How many tests should I need to perform ? The system meets the users' needs ... has 'the right system been built ? Sponsor Tester
$14,000 85% % Defects Introduced in this phase % Defects found in in this phase Percentage of Bugs $1000 $ Cost to repair defect in this phase $250 $130 $25 Coding Unit Test Funct Test Post Release Field Test Cost of testing in each test phase ? Defects costs Source: Applied Software Measurement, Capers Jones, 1996
Testing Fundamentals The purpose of testing is to evaluate products to: Compare the results against the requirements Help to resolve the differences Document the differences
Only when people involved in a common goal, use a common language and have a common understanding, can a project be successful.
What is testing ? Testing is the process of executing a program with intent of finding defects Traditional View
Quality Quality Planning Quality Planning phase: Agree with all project groups/team members, quality goals and objectives to be achieved by the project. Quality Management:Assure that all quality goals and objectives previously agreed in the Quality planning are achieved
Test Process Model Phases Test Project Management assure and demonstrate successful control and achievement of testing activities. Solution Change Management Assure that all changes will be implemented in an organized and controlled manner in order to minimize risks and to maximize benefits. Testing activities must be fully integrated within the change management process to ensure impacts on testing are assessed and managed Defect Management : Manage the tracking and fixing of all defects found during testing.
Test Process Model Phases Implementation: phase details the activities required to facilitate the physical deployment of the system into production. Testware Configuration ManagementManage and controll all system outputs (system funtion and associated tesware) in order to ensure the currency of deliverables is maintained. Tools and Test Environment Tools and Test Environment covers both the selection of appropriate test tools and the setting up of test environments to support one or more of the detailed test levels.
Types of Testing Static testingis a detailed examination of work products without execution the code. Work products such as requirements, statements, designs, test plans are “static tested” by using techniques, such as desk-checks, walk through and inspections. Static testing is applicable for all project phases. Early defect detection and prevention imply minor or no cost. Dynamic Testing Testing that is carried out by executing the code. Dynamic testing is a process of validation by exercising a work product and observing the behavior of its logic and its response to inputs.
Desck Check Checklist Review Mapping Prototyping Inspection Review
Levels of Testing UT - Unit testing IT- integration testing ST- System testing SIT- Systems integration testing UAT - User acceptance testing OT - Operability testing The Levels of Testing used in the application development lifecycle are:
The Basic Testing process Model MTP Master Test Plan
Test Planning Test Planning Test Planning is considered the most fundamental phase in securing a successful outcome to testing activities It provides the cornerstone for all subsequent testing activities During this phase, the scope of testing is documented in a Master Test Plan, or MTP, which is committed to by all groups. In addition, Detailed Test Plans, or DTP, are also documented and interlocked through the Master Test Plan for each level of testing, such as Unit test, System test, and Acceptance Test.
Test Planning Test Objectives How will the testing be performed? How will we know when testing is complete? • Entrance Criteria/Exit Criteria Requirements/Tractability What will be tested? • Functions/Features to be Tested • Functions/Features Not to be Tested Scheduling and Estimating When will the testing be conducted? Do we need training for this project ? Staffing Who will perform the test ?
Test Preparation Test Preparation The Test Preparation phase consists of a variety of activities that are completed to prepare each level of testing The Test Process Model advocates and enables test preparation activities to be carried out in parallel with other required development activities, presenting the opportunity for significantly reducing elapsed time.
Preparation Specification Creation of test design requirements, including purpose, preparation and usage. Cases Development of test objectives, including techniques and approaches for validation of product. Determination of the expected result for each test case. Scripts Documentation of the steps to be performed in testing. Focusing on the purpose and preparation of procedures; emphasizing entrance and exit criteria How will the testing be performed? Data Development of test inputs, determination of the data set needed to ensure a comprehensive test of the system.
The Key successful testing is test Planning Test Planning starts with the Test Strategy which provides a general direction for testing. The test strategy describes the complexity of the full-lifecycle testing approach that is used and helps to bring to the forefront the importance of a cost-effective, team approach to testing. The test strategy may be created as a generic strategy as soon as the solution requirements have been defined. With large Systems Integration type projects, a generic Test Strategy is included with the Statement of Work to emphasize the importance of test planning and to create an awareness of the size of the effort required If the Statement of Work does not include a generic test strategy, the strategy is the first major test work product that is developed and is folded into the Master Test Plan.
Master Test Plan Evolves from Test Strategy The Master Test Plan evolves from the Test Strategy and is driven by the quality goals contained in the quality plan The development of the Master Test Plan starts early in the development lifecycle, and is refined as more information becomes available. The Master Test Plan is a Document of Understanding. • the "what ", or scope and objectives • the "why ", or purpose • the "when ", or key milestones • the "who ", or organizational roles and responsibilities for all levels of dynamic testing
Organize Test Team Define Test Environment Requirements Define Test Objectives Determine Test Focus Develop Test Strategy Determine Types of Test and Select Test Levels Develop Test Schedules Develop Problem & Change Management Procedures Define Test Data Generation & Execution Procedures Define Test Reporting Procedures
Plan Prepare Execute Report
Execution Execute Tests Perform the activities necessary to execute tests in accordance with the test plan and test design. Compare actual x Expected Results Determine if the actual results met expectations. Test Log Logging tests in a desired form. This includes incidents not related to testing. Record Discrepancies Document defects as they happen including supporting evidence.
Report Summary Status report • Scope of test Test completion Criteria Code Coverage Requirement Coverage • Test Results Defect tracking and correction • What works /What does not works Metrics Defects/lines of codes Defects /Function Points • Recommendations Fix functions/interfaces that do not work.
Test Process Model Benefits • Technology Independent model • Independent of development lifecycle • Flexible and scaleable • Uses risk-based approach to testing • Reduced cost and cycle time • Improved test processes • Repeatable processes • Reusable test assets
? ? ? ? ?