SMU CSE 8314 / NTU SE 762-N Software Measurement and Quality Engineering

1. SMU CSE 8314 / NTU SE 762-NSoftware Measurement and Quality Engineering Module 05 Choosing a Maturity Level and Avoiding Stagnation

2. Is it Always Good to Move Higher in Maturity? • It depends on the organization • Its business • Its goals • Its resources • Its focus In the next few slides, we will discuss Weinberg’s method of analyzing an organization to determine the most appropriate level of maturity.

3. Weinberg’s Model for Maturity Analysis • What are the customer demands? • How difficult is it to sell them the product? • Are customer demands changing? Up or Down? • Are they becoming more demanding? • What are the problem demands? • How hard is it to make the product? I.e., to build the software? • Are they changing? High L4 is OK Level 5 Needed L3 is OK Customer Demand L2 is OK L1 is OK Low L0 is OK Low Problem Demand High

4. Mapping Present and Desired State Identify: Customer Demand level and Problem Demand level High L4 is OK Level 5 Needed L3 is OK Customer Demand L2 is OK L1 is OK Low L0 is OK Low Problem Demand High

5. Mapping Present and Desired State High L4 is OK Level 5 Needed L3 is OK Customer Demand L2 is OK Customer L1 is OK Low L0 is OK Low Problem Demand High Current

6. Mapping Present and Desired State High L4 is OK Level 5 Needed L3 is OK Customer Demand L2 is OK Customer L1 is OK Low L0 is OK Low Problem Demand High Current

7. Mapping Present and Desired State Anticipate: Changes in Demand level High L4 is OK Level 5 Needed L3 is OK Customer Demand L2 is OK Customer L1 is OK Low L0 is OK Low Problem Demand High New Current

8. Mapping Present and Desired State • Sophistication or complexity of a new product may require moving up a maturity level: • Change to a more mature process • or • Lower customer demand through lower price or lower expectations High L4 is OK Level 5 Needed L3 is OK Customer Demand L2 is OK Customer L1 is OK Low L0 is OK Low Problem Demand High New Current

9. Several Hypothetical Cases • To illustrate the process of deciding whether to increase process maturity • These are taken from Weinberg

10. Case A - Baseline • The original product sold 10,000 copies at $400 ea. It had minimal service costs. • The current release is generating equivalent revenue, but costs $10 per customer to service. • The next release could lower service costs by 20% or $2 per customer or $20,000. • But to handle the complexity of the next release we must spend $50,000 to upgrade our process. Upgrading maturity isn’t worth it unless the improvements will apply to future projects and enlightened management is willing to invest in the future

11. Case B - New Technology but Internal Impediment • The new version will save $2 per customer. At 10,000 customers, this is $20,000 saved • in the service manager’s budget. • New technology makes it possible to implement a more mature process for only $10,000 • but from the development manager’s budget! The internal inertia / impediment of the organizational structure may make this difficult to do, even though it benefits the organization as a whole.

12. Case C - New Technology with Strong Motivation • The new version will sell to 100,000 customers at a potential savings of $2 per customer or $200,000. In this case, the economic argument is so strong that it may be worth going to upper management to overcome organizational inertia.

13. Case D - Quality as a Motivator • Better quality would increase sales by 1000 copies at $400 per copy = $400,000 in income. • It will cost $50,000 to upgrade the process. • Therefore the improved quality is well worth it. If management believes it, this is a good case. Customer inputs helps in cases like this.

14. Case E - Cost of Quality is Not Justifiable • The new product will sell 10,000 copies at $4000 each = $40,000,000 • There is 1 chance in 4 of success. • It will cost us $10,000,000 to develop with our current process. • This “breakeven” proposition suggests that we need something different --- • either improve your process maturity, or • find a bigger market for the product, or • find an easier (lower risk) new product

15. Case F - Lower Your Maturity • We cannot find a way to produce the product at an acceptable cost. • So we will contract the development to a more mature subcontractor (specializing in software). • We will lower our process maturity (or keep it the same) and still keep our business. This is a good strategy if your product does not contain proprietary elements and software is only a secondary part of your business.

16. Case G - Gullible Customers / Reward for a Lousy Job • Our current product has many problems. • A fix will cost us money and customers will expect it free of charge. • Therefore, we will re-label the fixed version “release 5,” throw in a few cosmetic improvements, and charge them $50 for a “discounted” upgrade. • At 10,000 customers, this will net us $500K to correct the problem and make money too Undemanding customers, weak competition

17. Case H - Monopolies Lack Motivation • HAL corporation dominates the market. Customers are stuck with them because of compatibility. • HAL does not need to improve • “Why spend money to make it better?”

18. Case I - Competition can Topple Monopolies • HAL corporation was doing just fine until an upstart competitor came along with a better and fully compatible product. • By the time HAL got out a comparable product [2 years later], they had lost 50% of their market share. Short term stagnation can lead to long term ruin.

19. So Why Do Organizations Stagnate?

20. Circular Arguments for Stagnation - I “We can’t do better so we won’t try -- so we won’t find out if we can do better” [This is the classic discrimination argument used throughout history against various minorities or outsiders] • “xxxxx’s (probably) cannot do the job so we won’t educate them in how to do the job • so we won’t find out if they can do the job” {for xxxxx, substitute your favorite group}

21. Circular Arguments for Stagnation - II [maybe they never really tried] [maybe things have changed] “It will never work here because we are different” “We tried that once and it didn’t work”

22. Circular Arguments for Stagnation - III “We are already doing the best possible job -- so if others are doing better it must be easier for them” [“Everyone else has it easier than I do …”]

23. Circular Arguments for Stagnation - IV The Superhero Excuse “Our superhero is infallible - so if we fail, it is the fault of outsiders - or else it is impossible” “Our superhero is infallible - so ask him or her how to improve. Don’t listen to outsiders, who don’t understand the complexities of our business.”

24. Breaking the Circle • Method I - Threats of Intolerable Pain (for example, being put out of business by a competitor) • Example: US auto companies in the 1980’s • Method II - Enlightened and Effective Leadership • This is truly rare • Method III - Develop Overwhelming Evidence • This doesn’t always work, but it helps

25. Developing Evidence • Keep data on what actually happens • Record lessons learned, & distribute them • Especially keep data on success and failure rates • Compute the cost of failure • Benchmark with the competition and with other successful organizations Include the super-heroes and top management on the benchmarking visit

26. Data Collection is Hard • Poorly managed projects will rarely keep data on anything, especially for historical purposes, because it • Costs them time and money, but benefits someone else • Shows their warts and failings • And they don’t know what to collect anyway • So you must keep it for them • Even if they don’t know you are doing it

27. How to Avoid Historical Data 1 • Complain about the cost • Collection, storage, analysis • Morale of those being measured • Resist consistency (inconsistency makes data useless and hides flaws) • Emphasize why each subunit is unique • Collect different data at each subunit • Define data vaguely • Never spend time verifying the accuracy of the data 1 Advice for poor managers

28. Additional (more fundamental) Methods of Breaking the Circle • Establish technical reviews and invite many people outside the organization to attend • Send influential people to public seminars where they will see the work of others Hearing others brag is harder to ignore than any technical paper.

29. Another Method ofBreaking the Circle • Ask the organization, or influential people, “How would you spot a quality problem?” Then use their answer to find their quality problems and demonstrate them.

30. Low Maturity as a Power Hierarchy - I Level 0 - We trust only ourselves Level 1 - We don’t trust management Level 2 - We don’t trust programmers Trust is the key to success because it reduces the need for communication and data, too much of which can choke the organization.

31. Low Maturity as a Power Hierarchy - II Level 3 • Everyone is open to product information Level 4 • Everyone is open to process information Level 5 • Everyone is open to culture information

32. The Most Vicious Cycle “We are in deep trouble because we don’t know how to develop software. Therefore, we don’t have time to learn. Therefore, we are always in deep trouble.”

33. Examples of Inertia fromPast Success • We reuse obsolete but formerly successful code/processes (or people): • We have large volumes of successful code, which leads to high maintenance cost for obsolete software • BUT: Past success results in our unwillingness to change. • PROBLEM: Past success was in an old environment that is being replaced. People don’t always realize that the world has changed

34. Overcoming Inertia in a Successful Culture • All cultures strive to honor their past (maintain foundations), and retain present success • (I.e., nobody wants to slip backwards). • The trick is to also build toward the future.

35. Building Toward the Future Requires ... Humility - lack of arrogance Accurate Memory - about the true origins of the organization • Don’t be blinded by the myths A vision of the future - something that will inspire people

36. Necessary Changes in order to Move Up in Maturity 0 to 1 : Humility (from exposure to others) 1 to 2 : Ability (from training and experience) 2 to 3 : Stability (from quality management) 3 to 4 : Agility (from tools and methods and knowledge) 4 to 5 : Adaptability (from human development)

37. Another Vicious Cycle • We don’t have money to plan • We can’t waste time collecting data • We don’t have planning experience anyway • And don’t have money to learn These are symptoms of a poorly managed organization that is probably ripe for trouble

38. Cutting This Vicious Cycle - Option 1 • Enlightened leadership sees that this will lead to failure • You can try to educate the leadership through information about competitors, successes of others, etc. • You can become a leader and remember what you learned here

39. Cutting this Vicious Cycle - Option 2 Competition forces you to change • Customers can often have a big impact if they speak forcefully enough • Loss of business can also help • Pain is necessary to make change in the absence of enlightened leadership

40. The Problem of Excess Ambition OK, if we need to improve, why settle for level 2? Let’s move to level 3 or 4!

41. Ma, I don’t want to be an adolescent and then a teenager! I want to become an adult right away! The Analogy

42. The Problem • Each level of maturity acts as a foundation for the next level • And each involves fundamental changes in culture • If you skip a level or go through it too fast, it doesn’t “take” • The result is reversion to level 1 • And you may not even know it! (unconscious incompetence)

43. There Is One Way ... • Motorola Electronic Systems in India was able to start at CMM level 3 and move to level 5 • Hired all people with NO software background • No bad habits to “un-teach” • No culture to change • Started with the level 3 behaviors of the company’s best software group • Moved to level 5 in a few years

44. Summary • Understand the culture • See through the excuses • See both the outsider’s view and the insider’s view • Use models to guide your improvement process • Choose the appropriate level of target maturity • Be prepared to overcome vicious cycles

45. END OF MODULE 05