Software Quality Assurance Software Quality Assurance Lecture Notes M. D. Dykton 21 April 2003 University of Maryland Baltimore County CS 345 Software Engineering Software Quality Assurance
Outline • What is software quality? • What is software quality assurance? • How do you create software quality? Software Quality Assurance
Software Quality – Why Bother? • Importance of Software • Ever increasing levels of system functionality are embedded in software, not hardware • For complex, software-intensive systems • 1970’s…estimated functionality 20% software/80% hardware • 2000’s…estimated functionality 80%+ software and growing • Size, complexity and criticality of software is growing rapidly • Quality is not easily produced by accident anymore • Quality of work/reputation of organizations frequently depends upon producing quality software Software Quality Assurance
Quality – What is it? • The Institute of Electrical and Electronics Engineers' (IEEE) Standard Glossary of Software Engineering Terminology defines quality as “(1) the degree to which a system, component, or process meets specified requirements, and (2) customer or user needs or expectations.“ • Kitchenham states quality is "hard to define, impossible to measure, easy to recognize." Software Quality Assurance
Software Quality • Multi-faceted concept, not simply defined • Fundamentally, quality implies fitness for intended use • In part, implies software meets a specification or set of requirements • Also includes many additional attributes which may or may not be explicitly captured in a specification: • Correctness • Safety • Security • Reliability • Resilience • Robustness • Efficiency • Complexity • Reusability • Learnability • Usability • Testability • Understandability • Modifiability • Portability • Maintainability Software Quality Assurance
Software Quality Assurance • Quality Management System • Policy • Three Principal Elements of SQA: • Software Quality Assurance • Policy Establishment • Software Quality Planning • Software Quality Control • Quality Guidance • Standards • Practices & Procedures Process Refinement • Quality Controls • Process enforcement Feedback Project-specified plans developed from QA guidance … Project 1 QA Plan Project 2 QA Plan Project n QA Plan Software Quality Assurance
Software Quality Assurance Policy Establishment • High-level organization-based statement of software quality policy • Goals • Rationale • Relationship to other QA policies & standards • Specific policies and procedures • Roles and responsibilities • Organizational changes • Establishment of a Quality Assurance Management System Software Quality Assurance
Software Quality Assurance Organization – JHU/APL Example Steering Committee Laboratory-level management oversight, sets policy Software Engineering Process Group (SEPG) Laboratory-level group that defines policy Implementation and process improvement Department-level group that defines detailed policies, processes and procedures Quality Council orDept. SEPG Software Quality Assurance Manager (SQAM) Oversees and enforces department-level policies, processes and procedures; handles process tailoring, support infrastructure mgmt, staff training Mini-SQAMs or QA Teams Group-level QA support Software Quality Assurance
SQA Process Improvement • Software Engineering Institute (SEI) Software Capability Maturity Model (SW-CMM) – classifies five levels of process • Level 1 Initial – Ad hoc, unpredictable software process • Herding Cats…. • Level 2 Repeatable – Focus is on effective software project management processes for cost, schedule and functionality • Project planning • Project tracking and oversight • Requirements management • Quality Assurance • Configuration management • Subcontract management Software Quality Assurance
SQA Process Improvement(continued) • Level 3 Defined – Software management and engineering processes are standardized and documented • Emphasis is on formal procedures to ensure defined process is followed • Organizational product definition and focus • Software product engineering • Integrated software management • Reviews • Intergroup coordination • Level 4 Managed – Measures of the software process and product quality collected, and software process and products are quantitatively understood and controlled • Software quality management • Quantitative process management Software Quality Assurance
SQA Process Improvement(continued) • Level 5 Optimizing – Level 4 process coupled with planned and funded process improvement program • Process change management • Technology change management • Defect prevention Software Quality Assurance
Software Quality Planning • Software Policy & Procedure considerations • Risk management • Sound software management and engineering practices • Measurement program Software Quality Assurance
SQA and Risk Management • Risk management is an integral part of the process to develop software quality • Quality is not free…QA activities costs time and money…trade-offs are necessary • Quality assurance activities are risk reduction efforts • A “one-size-fits-all” quality assurance plan is rarely feasible • QA process tailoring is necessary • Risk management is an approach to intelligent process tailoring • Risk Management • Risk Identification • Risk Analysis and Assessment • Risk Planning & Mitigation • Risk Tracking Software Quality Assurance
SQA and Risk Management(continued) • SEI Software Risk Taxonomy • Use risk taxonomy as a checklist to identify risks and potential risk mitigation activities (i.e., QA process and products) • Program Constraints • Resources • Contract • Project Interfaces • Product Engineering • Requirements • Design • Code and Unit Test • Integration and Test • Engineering Specialties • Development Environment • Development Process • Development System • Management Process • Management Methods • Work Environment See Managing Risk: Methods for Software Systems Development, Elaine M. Hall, Addison Wesley Longman, Inc., 1998, p.76, Table 4.1 Software Quality Assurance
Software Quality Control • Process Enforcement • Independent agent (i.e., Software Quality Assurance manager/team) • Quantitative Metrics • Data Collection / Archiving • Review / Audit (process and products) • Authority to act • Process Assessment • Process Improvement Feedback Software Quality Assurance
Software Metrics • Necessary, underused and hard • Forms one of the pillars of “prove-able” quality • Software and its development is complex and multi-dimensional, hard to understand and measure • Attributes of “Good” Metrics • Useful • Meaningful • Quantifiable • Measurable • Repeatable • Metrics should be linked to risk mitigation activities and T&E program (both product and process) Software Quality Assurance
Software Metrics(continued - Example) Software Quality Assurance See Software Metrics: A Rigorous & Practical Approach, Norman E. Fenton and Shari Lawrence Pfleeger, PWS Publishing Company, 1997, p.76.
Software Metrics(continued) • Scope of software metrics – process, products or resources • Project management • Cost and level-of-effort estimation • Productivity measures • Quality attributes • Reliability measures • Performance • Defect tracking • Structural and complexity metrics • Software engineering process metrics Software Quality Assurance
Summary • Software Quality is important – software controls life-and-death decisions, has enormous economic consequences, affects reputations, etc. • An independent Quality Management System is vital component of an effort to produce quality software - Quality needs a “stakeholder” • “Complete” requirements determination is a critical first step, include both overt and implied requirements • Quality assurance is a product of effective risk management • Tailor project management and software engineering practices to mitigate quality-related risks • Measurement program – demonstrate progress toward quality objectives using metrics, testing and other measurements Software Quality Assurance
References • Managing Risk: Methods for Software Systems Development, Elaine M. Hall, Addison Wesley Longman, Inc., 1998. • Software Metrics: A Rigorous & Practical Approach, Norman E. Fenton and Shari Lawrence Pfleeger, PWS Publishing Company, 1997. • Software Quality, Theory and Management. Gillies, Alan C., International Thomson, Computer Press, 1997. • Risk Mangement Processes for Software Engineering Models, Marian Myerson, Artech House, 1996. • Software Engineering, Ian Sommerville, 5th Edition, Addison-Wesley Publishing Company, 1996. • Software Quality Assurance: A Practical Approach, Ernest Wallmuller, Prentice Hall International (UK) Ltd., 1994. • ISO 9001 and Software Quality Assurance, Darrel Ince, McGraw-Hill Book Company, 1994. • Handbook of Software Quality Assurance, Edited by G. Gordon Schulmeyer and James I. McManus, 2nd Edition, Van Nostrand Reinhold Company, 1992. • Managing the Software Process, Watts S. Humphrey, Addison-Wesley Publishing Company, 1989/1990. • Applications Strategies for Risk Analysis, McGraw-Hill Software Engineering Series, Robert N. Charette, Intertext Publications, 1990. • Software Product Assurance: Techniques for Reducing Software Risk, William L. Bryan and Stanley G. Siegel, Elsevier Science Publishing Co., Inc., 1988. • Characteristics of Software Quality, Barry W. Boehm, John R. Brown, Hans Kaspar, Myron Lipow, Gordon J. MacLeod and Michael J. Merritt, North-Holland Publishing Company, 1978 • Kitchenham, Barbara, and Shari Lawrence Pfleeger. "Software Quality: The Elusive Target." IEEE Software 13, 1, Jan. 1996: 12-21. Software Quality Assurance
