A Simple View of How to Reduce Total Cost of Quality presented by Catherine Oehl

1. A Simple View of How to Reduce Total Cost of Qualitypresented by Catherine Oehl COQAA Monthly Meeting December 5, 2012

2. AGENDA • What is Quality and Why is it important • What is Total Cost of Quality (TCoQ) • What is the purpose and value of measuring TCoQ • How to measure and reduce TCoQ • Hypothetical Scenario applying TCoQ • Q&A

3. Quality Defined • Definition: A measure of excellence or a state of being free from defects, deficiencies and significant variations. It is brought about by strict and consistent commitment to certain standards that achieve uniformity of a product in order to satisfy specific customer or user requirements. • ISO 8402-1986 standard defines quality as "the totality of features and characteristics of a product or service that bears its ability to satisfy stated or implied needs." • If an automobile company finds a defect in one of their cars and makes a product recall, customer reliability and therefore production will decrease because trust will be lost in the car's quality. Source: http://www.businessdictionary.com/definition/quality.html

4. Consequences of poor quality

5. Common State of Quality • Current projects spend 40-50% of the effort on avoidable rework • About 80% of avoidable rework comes from 20% of the defects • About 80% of defects comes from 20% of the modules and about half of the modules are defects free • 90% of the downtime comes from at most 10% of the defects • Peer Reviews catch 60% of the defects • Perspective-based reviews catch 35% more defects than non directed reviews • Disciplined personal practices can reduce defect introduction rates by up to 75% • While several organizations measure and track application total cost of ownership (TCO), very few measure the total cost of quality

6. WHAT IS TCOQ • Total Cost of Quality (TCoQ) consists of the costs associated with preventing, assessing, finding and correcting defective work. • Note that the TCoQ analysis looks only at our (producers) costs and not the customer’s cost due to poor quality.

7. QUALITY COSTS of TCoQ Prevention cost + Inspection cost + Internal Failure cost + External Failure cost • Prevention costs arise from efforts to keep errors or defects from occurring at all • Inspection costs arise from detecting errors/defects via inspections, tests, audits • Internal failure costs arise from defects caught internally and dealt with by discarding or repairing the defective items • External failure costs arise from defects that actually reach customers. Often other costs are incurred in addition to correcting the failure.

8. Purpose and value of TCOQ Addresses and measures the following questions: • What is the state of quality of your applications? • How does quality impact your business? • How is quality measured? • When are errors/defects introduced? Why? • When are errors/defects corrected? What are the costs? • What are the processes that impact quality? • How do you quantify the effectiveness of each process? • How much do failures cost? • What is the root cause of the failure?

9. When and where are defects introduced Source: Writing Testable Requirements, Dick Bender • Source: NIST Software Quality Study

10. When and where are defects Detected • Source: NIST Software Quality Study

11. Cost to Correct Defects Cost to remove defects is 100 times more • Source: B. Boehm and V. Basili, “Software Defect Reduction Top 10 List”, IEEE Computer

12. The quality Lever Prevention / Reviews Cost Savings Appraisal / Inspection Failures / Reaction Greatest Leverage Good Leverage Least Leverage Source: http://en.wikipedia.org/wiki/Quality_costs

13. Preventive Costs GOAL: Prevent defects before they happen (Work done right the first time) Recommended Activities: • Formal Requirements Gathering Processes • Requirements Analysis and Management • Early Prototyping • Usability Analysis • Static Testing • Peer Reviews • Release and Configuration Management Processes • Standards for Test Environments • Quality Training • Accurate Internal Communication

14. inspection costs GOAL: Detect the Errors (Find them early) Recommended Activities: • Code inspections • Design reviews • Glass box and black box testing • Test Automation • Usability Testing • Pre-release / User Acceptance Testing • Structured Personal reviews focused on verification and validation

15. Internal failure costs GOAL: Report and fix defects before production release (Fix and Re-Test) Incurred Activities: • Diagnose Defects • Defect Fixes • Defect Management • Re-Test and Regression Testing • Rework code and recompile • Wait states

16. external failure costs GOAL: Reduce and remediate impact to customers (Apologize and recover) Incurred Activities: • Technical Support Calls • Recalls • Customer Service • Scrap • Overtime • Regulatory / Legal Costs

17. HYPOTHETICAL Scenario • Suppose we have a software product that has one production release per quarter. • On average each release contains 1000 must-fix defects that will be identified and repaired. • We will assume developers catch 25% of defects during unit and integration testing at a cost of $10/defect

18. 1 - no Inspection Testing

19. 2 - Manual Inspection Testing

20. 3 - Manual with Automated Inspection Testing What if we added some simple Prevention activities like Static Reviews, Peer Reviews, Change Management ?

21. 4 - Adding Prevention Activities

22. Summary • Quality creates value and reduces risk exposure • The “cost of quality” isn’t the price of creating a quality product or service. It’s the cost of NOT creating a quality product or service. • Errors / defects are often introduced early and typically found later • Total Cost of Quality (TCoQ) consists of the costs associated with preventing, assessing, finding and correcting defective work. • The central theme of quality improvement is that larger investments in prevention drive even larger savings in quality-related failures and Inspection efforts. • Any Quality improvement effort should use TCoQ to quantify the effectiveness of the process.

23. Contact Info and Q&A Catherine Oehl oehlc@yahoo.com 614-440-6477 LinkedIn Profile: http://lnkd.in/u8kcyy

24. Resources & References • Software Defect Reduction Top 10 List by Barry Boehm and Victor Basili • Using Cost Benefit Analysis to Develop Software Process Improvement Strategies, Data Analysis Center of Software • Traditional P-A-F method by Juran (1951) and Feigenbaum (1956) – Prevention Appraisal and Failure costs • Phil Crosby’s Model from the book Quality is Free (1979) – conformance to requirements – cost of conformance (done Right) and non-conformance (done Wrong) • Process Cost Model by Ross (1977) • http://asq.org/learn-about-quality/cost-of-quality/overview/overview.html • http://www.authorstream.com/Presentation/Hassanasif-493959-cost-of-quality • http://www.bexcellence.org/Cost-Of-Quality.html • http://www.isixsigma.com/implementation/financial-analysis/cost-quality-not-only-failure-costs/ • http://www.projectconnections.com/articles/051005-koch.html

25. Suggested TCoQ Benchmark Measurements • Process (Productivity & Prevention) • Effectiveness of Test Processes • Release Mgt, Configuration Mgt • Efficiency of Test Processes • Defect Mgt, Test Environment Mgt, Test Data Mgt • Consistency of Test Activities • Test Planning, Test Estimation, Test Execution • Schedule • Testing Planning • Duration of lifecycle phases • Duration of Test Phase/Stage • Test Estimating/Projections • Planning Estimate • Component Based Estimates

26. Common Measurements of Failures • Failures • Volume of Defects • Open, Closed, New, Test Phase/Stage Contained, Defect Aging (# of days open) • Severity of Defects • Criticals, Highs, Mediums, Lows, Problem Areas, Low to all reported ratio

27. Simple Actual quality Costs • Build Test Mgt Costs (P+I+IF) • Manual Test Hour Costs • Automated Test Hour Costs • Test Hour Costs per Test Phase/Stage • % of Total Build Costs • Breakfix/Maintenance Costs (EF)

28. View of TCOQ in terms of QA