SMU CSE 8314 / NTU SE 762-N Software Measurement and Quality Engineering

1. SMU CSE 8314 / NTU SE 762-NSoftware Measurement and Quality Engineering Module 11 Analyzing the Net Cost of a Process

2. Analyzing the Net Cost of a Process • ) Identify and list all steps or tasks of the process • Document each step -- purpose, description, procedures, etc. • Include everything you do. • This works best if you watch and record actual behavior of practitioners.

3. Analyzing the Net Cost of a Process • ) For each task, do a basic value-added analysis: • Divide all tasks into value-added and non-value-added • A task is “value-added” if it meets the three strict criteria: • Changes the product • The change is wanted by the customer • It is done right the first time (is not rework)

4. Scrutinize Value-Added Tasks • Inputs • What is their source? • From whom or what? (supplier). • Is there an alternative? • Does the supplier understand my dependence on them? • What value does it add, if any? • Why do this? For whom? • What is the impact if I do not do this? • Do they care that I do this?

5. Scrutinize Value-Added Tasks (continued) • Outputs • What do I produce? • To Whomdoes it go? (i.e., who is the customer of this task?) • Do they really need it? You may find that at least some portion of these tasks is really non-value-added

6. For the Moment, we Ignore Whether the Non-Value-Added Tasks are Essential • We can take that into consideration later, when deciding which tasks to focus on eliminating or streamlining • But it is a good idea to scrutinize the “essential” tasks to make sure they are really essential

7. Scrutinize Value-Added-Essential Tasks • Why must we do it? • What happens if we don’t do the task? • Does anyone really care? • Would there be a genuine problem? • What does the task accomplish? • What would happen if we did it a different way? • Can it be simplified or streamlined? • Is all of it essential?

8. Sample Result of Analysis Value-Added Non-Value-Added (costs $, no value to customer) 1) Customer Wants 2) Changes Product 3) Done Right the First Time Essential Non-Essential • Estimating • Training • Planning • Customer-required acceptance test • Configuration Control • Inspections • Approval by 7 people! • Delays for test systems • Data conversion between design tool and coding tool • Wait for subcontracted hardware • Debugging • Service calls • Warranty costs • Fedex costs for patches • Loss of customer goodwill • etc. • Requirements analysis • Design • Coding • Documentation • Integration • Manufacturing • Packaging • Shipping

9. Net Cost of a Process(continued) 3) Divide into five categories: A) Costs of performing the process • value-added B) Prevention costs • non-value-added, cost of quality (conformance) C) Appraisal or evaluation costs • non-value-added, cost of quality (conformance) D) Failure response costs • non-value-added, cost of quality (non conformance) E) Everything else • Non-value-added, non cost of quality

10. Net Cost of a ProcessCategorizing Tasks & Subtasks Non Cost of Quality Cost of Quality (all non-value-added) Cost of Conformance Cost of Non Conformance Non- Value- Added Value Added Prevention Appraisal Failure • Design • Development • Fabrication • Documentation • Assembly • Process • Creation • Upgrade • Shipping • Over- • head • Errors • Ineff- • icien- • cies • Training • Planning • Simulation • Modeling • Consulting • Qualifying • Certifying • Inspection • Testing • Audits • Monitoring • Measure- • ment • Verification • Analysis • Rework • Service • Modification • Expediting • Recall • Correction • Retest • Error Analysis

11. Some Costs Fit Multiple Categories • Example: • The design process is mostly value-added but about 20% of the cost is for a design inspection which is, technically, non-value-added • If you break the process into more detailed sub-processes, you can usually assess these situations more accurately

12. Net Cost of a Process(continued) 4) Determine costs • Determine the cost in labor or other units for each task in each category • Sum for all tasks. % of time spent Task ___ % aaa 12.5 bbb 8.0 … … Time Card Labor Hours Spent

13. Example - Cost of a Process Cost of Quality Non Cost of Quality Cost of Non Conformance value-added Cost of Conformance Non-value-added TOTAL 600 100 450 95 300 450 400 275 200 150 150 3170 100% Prevention Appraisal Failure Program Design Design Reviews Code Code Reviews Unit Test Integration Test System Test Configuration Mgt Problem Solving Rework Retest 540 400 940 30% 60 50 110 3.5% 60 60 120 4% 40 35 300 450 400 275 1500 45% 200 150 150 500 15%

14. What if I Don’t Know? • Often at this point you have tasks you do not know • Cost of some tasks • Value of some tasks • Quality of some tasks • Productivity or cycle time of some tasks • Which leads us to step 5 - defining a metrics program to help with this analysis

15. Net Cost of a Process(continued) 5) For each task, determine: • Customer requirements (input & output) • Supplier requirements (input & output) • Measures that determine if requirements are met • Measures that determine why or why not • Other measures needed to assess value or cost

16. Net Cost of a Process(continued) 6) For each task: • Analyze the Measurement Requirements • Is it measured now? • Can it be measured? • Should it be measured? • Determine the level of effort or cost to measure

17. Use the Measures to Determine ... • What tasks cost the most • What tasks are worth improving • What tasks should be eliminated

18. Net Cost of a Process (continued) • ) Look for opportunities to improve • Tasks that are not value-added but have high cost • Tasks that have high cost of non-conformance This is where we waste so much time!

19. Comments • As the process matures, the trend is to have more and more prevention costs and fewer appraisal & failure costs • Your process maturity may have a lot to do with which techniques work and which do not!

20. Rework • One of the first things to measure is rework • Easy to explain to upper management • Contributes negatively to cost, quality & cycle time

21. Rework Loops Defect Detected Process Step Process Step Undetected Defect Several Steps

22. Rework costs money and time Doing it Over Again Defect Detected Process Step Process Step Undetected Defect Several Steps

23. Comments (continued) • By tracking these costs and the impact of process changes, we can determine the cost and the benefit of changes • Thus we can optimize the process • We can optimize for: • Quality • Cost, or • Cycle Time • Process engineering can help improve all three!

24. Example of Tracking Project Gamma Project Beta Project Alpha

25. More Comments • If we track current costs • We open our eyes to the cost of non conformance • If we track the effect of changes • We learn what works to improve our process

26. Cost of Quality vs. Value-Added • The cost of quality chart helps us identify which costs contribute to cost of quality and which costs contribute to cost of failure to have quality • Note that both charts put value-added tasks on the left • But cost of quality analysis helps us decide which non-value-added costs to focus on and where to add non-value-added tasks (cost of conformance) to reduce net costs

27. Comparison Similar Cost of Quality Concept Value-Added Concept Differences VA: value-added Tasks NCOQ: Non-Cost of Quality Tasks Some tasks may add neither value nor quality Non-Value-Added Essential Tasks Cost of Conformance Tasks Some tasks help quality but are not essential Non-Value-Added Non-Essential Tasks Cost of Non-Conformance Tasks Some NVA tasks do not address problems

28. Measuring “Cost of Quality” • Money is the language of management • You need to tell them how much quality costs • You need to show them that improvements are possible in terms of net cost • (In some organizations, other things matter, such as overhead rates, people levels, prestige, etc.)

29. Measure Actual Costs and Other Data • Record actual data in a historical database • Labor • Material • Cost of quality vs. non cost of quality • Cost of conformance vs. cost of non conformance • Project future costs on the basis of historical precedent

30. Analyze Actual Costs and Other Data • Compare alternative approaches • Process changes • Process differences • Other differences

31. What you think is happening What is really happening What you think should happen Humphrey’s Model Measurement can help you understand what is really happening

32. Beware ... Use process costs to measure processes, not people • Or people will change performance to make the numbers look good • Or you will become overly dependent on having only the best people • Success only because you have the best people does not constitute genuine organizational capability improvement

33. Process Changes Have Many Benefits • Consistency • Long term impact • Not overly dependent on individuals BUT • This does not mean that good people are not valuable • A good process enables • Best people to maximize their contributions • The organization to function smoothly when personnel changes occur

34. Summary - I • Understanding value is the starting point for effective and efficient processes • Value-added analysis can help identify the best places to focus improvement efforts • Software quality analysis is different from hardware quality analysis because the processes are different. Software is more dependent on development cost than production cost. • Quality tasks can be divided into cost of conformance and cost of non-conformance

35. Summary - II • Cost of conformance can be divided into cost of prevention and cost of detection • Invest in cost of conformance to reduce cost of non-conformance • Invest in prevention to reduce overall costs • Tracking of costs can demonstrate the true costs of non-conformance, especially of rework • Cost of quality and value-added are similar concepts but not identical

36. References • Crosby, Philip B. Quality is Free, New York, McGraw-Hill, 1979. • Deming, W. Edwards, Out of the Crisis, MIT Press, 1986, ISBN: 0911379010 • Juran, J. M., Juran on Leadership for Quality: An Executive Handbook, The Free Press, 1989.

37. END OF MODULE 11