Software Quality Assurance Lecture Notes M. D. Dykton 21 April 2003 University of Maryland Baltimore County CS 345 Soft

1. Software Quality Assurance Software Quality Assurance Lecture Notes M. D. Dykton 21 April 2003 University of Maryland Baltimore County CS 345 Software Engineering Software Quality Assurance

2. Outline • What is software quality? • What is software quality assurance? • How do you create software quality? Software Quality Assurance

3. Software Quality – Why Bother? • Importance of Software • Ever increasing levels of system functionality are embedded in software, not hardware • For complex, software-intensive systems • 1970’s…estimated functionality 20% software/80% hardware • 2000’s…estimated functionality 80%+ software and growing • Size, complexity and criticality of software is growing rapidly • Quality is not easily produced by accident anymore • Quality of work/reputation of organizations frequently depends upon producing quality software Software Quality Assurance

4. Quality – What is it? • The Institute of Electrical and Electronics Engineers' (IEEE) Standard Glossary of Software Engineering Terminology defines quality as “(1) the degree to which a system, component, or process meets specified requirements, and (2) customer or user needs or expectations.“ • Kitchenham states quality is "hard to define, impossible to measure, easy to recognize." Software Quality Assurance

5. Software Quality • Multi-faceted concept, not simply defined • Fundamentally, quality implies fitness for intended use • In part, implies software meets a specification or set of requirements • Also includes many additional attributes which may or may not be explicitly captured in a specification: • Correctness • Safety • Security • Reliability • Resilience • Robustness • Efficiency • Complexity • Reusability • Learnability • Usability • Testability • Understandability • Modifiability • Portability • Maintainability Software Quality Assurance

6. Software Quality Assurance • Quality Management System • Policy • Three Principal Elements of SQA: • Software Quality Assurance • Policy Establishment • Software Quality Planning • Software Quality Control • Quality Guidance • Standards • Practices & Procedures Process Refinement • Quality Controls • Process enforcement Feedback Project-specified plans developed from QA guidance … Project 1 QA Plan Project 2 QA Plan Project n QA Plan Software Quality Assurance

7. Software Quality Assurance Policy Establishment • High-level organization-based statement of software quality policy • Goals • Rationale • Relationship to other QA policies & standards • Specific policies and procedures • Roles and responsibilities • Organizational changes • Establishment of a Quality Assurance Management System Software Quality Assurance

8. Software Quality Assurance Organization – JHU/APL Example Steering Committee Laboratory-level management oversight, sets policy Software Engineering Process Group (SEPG) Laboratory-level group that defines policy Implementation and process improvement Department-level group that defines detailed policies, processes and procedures Quality Council orDept. SEPG Software Quality Assurance Manager (SQAM) Oversees and enforces department-level policies, processes and procedures; handles process tailoring, support infrastructure mgmt, staff training Mini-SQAMs or QA Teams Group-level QA support Software Quality Assurance

9. SQA Process Improvement • Software Engineering Institute (SEI) Software Capability Maturity Model (SW-CMM) – classifies five levels of process • Level 1 Initial – Ad hoc, unpredictable software process • Herding Cats…. • Level 2 Repeatable – Focus is on effective software project management processes for cost, schedule and functionality • Project planning • Project tracking and oversight • Requirements management • Quality Assurance • Configuration management • Subcontract management Software Quality Assurance

10. SQA Process Improvement(continued) • Level 3 Defined – Software management and engineering processes are standardized and documented • Emphasis is on formal procedures to ensure defined process is followed • Organizational product definition and focus • Software product engineering • Integrated software management • Reviews • Intergroup coordination • Level 4 Managed – Measures of the software process and product quality collected, and software process and products are quantitatively understood and controlled • Software quality management • Quantitative process management Software Quality Assurance

11. SQA Process Improvement(continued) • Level 5 Optimizing – Level 4 process coupled with planned and funded process improvement program • Process change management • Technology change management • Defect prevention Software Quality Assurance

12. Software Quality Planning • Software Policy & Procedure considerations • Risk management • Sound software management and engineering practices • Measurement program Software Quality Assurance

13. SQA and Risk Management • Risk management is an integral part of the process to develop software quality • Quality is not free…QA activities costs time and money…trade-offs are necessary • Quality assurance activities are risk reduction efforts • A “one-size-fits-all” quality assurance plan is rarely feasible • QA process tailoring is necessary • Risk management is an approach to intelligent process tailoring • Risk Management • Risk Identification • Risk Analysis and Assessment • Risk Planning & Mitigation • Risk Tracking Software Quality Assurance

14. SQA and Risk Management(continued) • SEI Software Risk Taxonomy • Use risk taxonomy as a checklist to identify risks and potential risk mitigation activities (i.e., QA process and products) • Program Constraints • Resources • Contract • Project Interfaces • Product Engineering • Requirements • Design • Code and Unit Test • Integration and Test • Engineering Specialties • Development Environment • Development Process • Development System • Management Process • Management Methods • Work Environment See Managing Risk: Methods for Software Systems Development, Elaine M. Hall, Addison Wesley Longman, Inc., 1998, p.76, Table 4.1 Software Quality Assurance

15. Software Quality Control • Process Enforcement • Independent agent (i.e., Software Quality Assurance manager/team) • Quantitative Metrics • Data Collection / Archiving • Review / Audit (process and products) • Authority to act • Process Assessment • Process Improvement Feedback Software Quality Assurance

16. Software Metrics • Necessary, underused and hard • Forms one of the pillars of “prove-able” quality • Software and its development is complex and multi-dimensional, hard to understand and measure • Attributes of “Good” Metrics • Useful • Meaningful • Quantifiable • Measurable • Repeatable • Metrics should be linked to risk mitigation activities and T&E program (both product and process) Software Quality Assurance

17. Software Metrics(continued - Example) Software Quality Assurance See Software Metrics: A Rigorous & Practical Approach, Norman E. Fenton and Shari Lawrence Pfleeger, PWS Publishing Company, 1997, p.76.

18. Software Metrics(continued) • Scope of software metrics – process, products or resources • Project management • Cost and level-of-effort estimation • Productivity measures • Quality attributes • Reliability measures • Performance • Defect tracking • Structural and complexity metrics • Software engineering process metrics Software Quality Assurance

19. Summary • Software Quality is important – software controls life-and-death decisions, has enormous economic consequences, affects reputations, etc. • An independent Quality Management System is vital component of an effort to produce quality software - Quality needs a “stakeholder” • “Complete” requirements determination is a critical first step, include both overt and implied requirements • Quality assurance is a product of effective risk management • Tailor project management and software engineering practices to mitigate quality-related risks • Measurement program – demonstrate progress toward quality objectives using metrics, testing and other measurements Software Quality Assurance

20. References • Managing Risk: Methods for Software Systems Development, Elaine M. Hall, Addison Wesley Longman, Inc., 1998. • Software Metrics: A Rigorous & Practical Approach, Norman E. Fenton and Shari Lawrence Pfleeger, PWS Publishing Company, 1997. • Software Quality, Theory and Management. Gillies, Alan C., International Thomson, Computer Press, 1997. • Risk Mangement Processes for Software Engineering Models, Marian Myerson, Artech House, 1996. • Software Engineering, Ian Sommerville, 5th Edition, Addison-Wesley Publishing Company, 1996. • Software Quality Assurance: A Practical Approach, Ernest Wallmuller, Prentice Hall International (UK) Ltd., 1994. • ISO 9001 and Software Quality Assurance, Darrel Ince, McGraw-Hill Book Company, 1994. • Handbook of Software Quality Assurance, Edited by G. Gordon Schulmeyer and James I. McManus, 2nd Edition, Van Nostrand Reinhold Company, 1992. • Managing the Software Process, Watts S. Humphrey, Addison-Wesley Publishing Company, 1989/1990. • Applications Strategies for Risk Analysis, McGraw-Hill Software Engineering Series, Robert N. Charette, Intertext Publications, 1990. • Software Product Assurance: Techniques for Reducing Software Risk, William L. Bryan and Stanley G. Siegel, Elsevier Science Publishing Co., Inc., 1988. • Characteristics of Software Quality, Barry W. Boehm, John R. Brown, Hans Kaspar, Myron Lipow, Gordon J. MacLeod and Michael J. Merritt, North-Holland Publishing Company, 1978 • Kitchenham, Barbara, and Shari Lawrence Pfleeger. "Software Quality: The Elusive Target." IEEE Software 13, 1, Jan. 1996: 12-21. Software Quality Assurance