Software Verification and Validation (V&V)

1. Software Verification and Validation (V&V) By Roger U. Fujii Presented by Donovan Faustino

2. Introduction • The paper has these following topics. • Software V&V Concepts • Planning for V&V • Life Cycle V&V Tasks • V&V Techniques and Methods • Measurements Applied to V&V • What is V&V?

3. Definitions • V&V – a system engineering discipline employing a rigorous methodology for evaluating and assessing the correctness and quality of software throughout the software life cycle. • Verify a developers process is technically sound.

4. V&V and QA • V&V and QA are not the same, but compliment each other. • V&V usually focuses on ensuring the requirements are being met, the overall project is focused on the correct objectives, and risk is being managed. • QA is focused on the day to day aspects of a project and is used to determine if procedures are followed

5. V&V Concepts • Benefits of V&V • Early detection leads to a better solution rather than quick fixes • Validating the solution is solving the “right problem” against software requirements • Objective evidence of software and system compliance to quality standards • Support process improvements with an objective feedback on the quality of development process and products

6. Field of Application • Important to conduct V&V in its interactions with the system of which it is part. • V&V examines the interaction with each of the key system components. • The V&V process addresses the following interactions with software environment, operators/users, hardware, and other software.

7. Planning for V&V • Scope of work • Software Integrity Levels • Development of the Software V&V Plan (SVVP) • Cost of V&V

8. Planning for V&V (cont) • V&V is more effective when initiated during the acquisition process and throughout the life cycle of the software. • V&V has importance levels or called “Integrity Levels” • Example • Medical device – high level • Personnel record-keeping system – low level

9. Integrity Levels • The level is a range of values that represent software complexity, criticality, risk, safety level, security level, desired performance, reliability, or other project-unique characteristics. • Each level defines the minimum required V&V tasks. • ANSI/IEEE Std 1012 defines four levels. Level 4 is assigned to high-assurance or critical systems

10. Development of the Software V&V Plan (SVVP) • How the V&V process is to be accomplished • Pg 36 gives a list. 13 key steps.

11. Cost of V&V • How much software V&V is adequate? • Rule of thumb for estimating V&V efforts ranges 10% - 33% depending on the software integrity level. • 33% - life sustainig medical devices or nuclear weapons • 20-25% - telecommunications or financial systems • 10-18% systems desiring software quality but not high-integrity

12. Life Cycle V&V Tasks • Pg 37-39 explains what V&V tasks go through the life cycle of the project • Goes throughout the life cycle of V&V starting from the Acquisition phase and all the way to the Maintenance phase. • The tasks listed in the book is defined from the ANSI/IEEE Std. 1012.

13. Life Cycle V&V Tasks • Acquisition V&V • Supply V&V • Development V&V (Concept, Requirements, Design, Implementation) • Development V&V (Test) • Development V&V (Installation and Checkout) • Operation V&V • Maintenance V&V

14. V&V Techniques and Methods • Audits, Reviews, and Inspection • Analytic Techniques • Dynamic Techniques

15. Audits, Reviews, and Inspection • V&V use these techniques to verify the software during its development process • Peer Reviews • Documentation inspections • Requirements/design/code reading • Test witnessing • Installation audits

16. Analytic Techniques • Static analysis of the software (i.e, requirements, design, or code) using graphical, mathematical formulas or diagrams. • Effective in error detection at the software unit level

17. Analytic Techniques • Control (data) flow diagramming • Interface input/output/process diagramming • Algorithm and equation analysis • Database analysis • Sizing and timing analysis • Proof of correctness

18. Dynamic Techniques • Involve the execution of the software. • Effective at error detection when software units are combined at the integrated subsystem and system levels

19. Dynamic Techniques • Simulation and modeling • Hardware/software benchmark testing • Hardware-in-the loop testing – the system config. is heavily instrumented to simulate different test scenarios to be created. • Scientific testing – coding of the target requirements/design using a general-purpose computer and higher order language.

20. Measurement Applied to V&V • Uses various calculated measurements to determine when the analysis or testing is completed, where errors are mostly likely to occur in the software, and what development process or function is causing the largest number of errors. • Based on these measurements, the software engineer can determine where to concentrate their efforts.

21. Measurement Methods • Software Structural Metrics – measures pinpoint program logic having greater logical or data complexity • Statistics-Based Measurements – examines program error rates, categorization of errors, and error discovery time periods • Trend Analysis – analyzing percent of errors with historical data • Prediction-Based Measurement – using reliability models to determine how much analysis and test effort to be done.

22. Conclusions • The V&V methodology and measurements are outlined in ANSI/IEEE Standard 1012. • Provides the framework for achieving an effective V&V effort • V&V is part of the software quality management process as defined in the IEEE SWEBOK. • Complimentary to and supportive of the software quality assurance, reviews, and inspections.

23. Questions?