Personal Software Process Software Quality

1. Personal Software ProcessSoftware Quality CIS 376 Bruce R. Maxim UM-Dearborn

2. These notes are based on: Introduction to the Personal Software Process Watts S. Humphrey Addison-Wesley Longman (1997)

3. Software Quality • Product meets the customer’s needs (not necessarily the same as customer’s wants) • Defects are the primary measure of quality in PSP • Yield Management • defect removal • defect prevention

4. Hierarchy of Needs • Software performs its required tasks • Product meets its performance requirements • Software is usable • Development is economical and timely • Product is dependable and reliable

5. Software Bugs • Latent bugs must • be operationally insignificant • not be destructive • not be observed often • Bugs are not important to the customer if they do not • affect operations • cause inconvenience • cost time or money • cause loss of confidence in software results

6. PSP Quality Focus • Low defect content is an essential prerequisite to quality software process • Low defect rates can best be achieved at the PSP level (e.g. individual SE’s) • SE’s are the ultimate source of defects are injected • SE’s need to remove defects, determine their causes, and learn to prevent them

7. Testing and Inspections • The sooner a defect is located the cheaper it is to repair and faster the repair • Design inspections will • improve product quality • lead to a more predictable schedule • allow more time to focus on quality issues • It is important for SE’s to review their own work before giving it to others to review

8. Some Fix Time Data • Some typical fix time ratios • IBM rules of thumb - coding: 1.5; testing: 60; usage: 100 • Boehm - design: 1; development test: 15 to 40; acceptance test: 30 to 70; operation: 40 to 1000 • Remus - design: 1, code: 20, test: 82 • Ackerman - test: 2 - 10 times inspection time • Russell - inspection: 1, test: 2 to 4, use: 33 • PSP research - unit test takes 12 times longer than code review to find and fix defects

9. Cost of Quality - part 1 • Failure costs • repair, rework, scrap • in PSP includes all compile and testing time • Appraisal costs • cost of defect inspections • in PSP includes all design and code review time

10. Cost of Quality - part 2 • Prevention costs • finding and resolving defect causes • handle before project starts • should be a process (not product) activity • in PSP this includes • formal specification and design work • prototyping • process analysis and improvement

11. PSP Quality Strategy - part 1 • Identify PSP quality objectives, e.g. • remove all defects before first compile • Establish PSP process quality measures, e.g. • overall process yield • LOC reviewed per hour • Examine products reviewed • determine their ratings on the measures • see which behaviors impacted these results

12. PSP Quality Strategy - part 2 • Based on these data, identify your most effective work practices. • Incorporate these practices in your process artifacts • process scripts • checklists • forms • Identify measures that predict process quality • use these as control variables • set specifications for these variables

13. PSP Quality Strategy - part 3 • Track your performance against these specifications. • Track your process to determine • when and if to change these specifications • actions to take for further process improvement

14. Appraisal to Failure Cost Ratio • Appraisal COQ = 100*(design review time + code time)/ (total development time) • Failure COQ (Cost of Quality) = 100*(compile time + test time)/(total development time) • A/FR (Appraisal to Failure Cost Ratio) ratio = (Appraisal COQ)/(Failure COQ)

15. Yield vs. A/FR • High A/FR ratios appear to lead to higher yields • 70+% yields not achieved without A/FRs near 1.0 or above • high A/FR does not guarantee high yield - the appraisal time must be spent effectively

16. Test Defects vs. A/FR • Defects are reduced by high A/FR ratios • To get very low numbers of test defects, A/FR values of above 2.0 appear required. • With A/FRs between 1.0 and 2.0, low test defects are occasionally obtained. • With an A/FR of below 1.0, low test defects are rare.

17. Yield and A/FR vs. Productivity • Productivity has considerable variation among individuals. • In some cases, high yield produces higher productivity but in others it does not. • High A/FR also sometimes results in high productivity and sometimes not. • Yield and A/FR are somewhat independent of productivity.

18. Process Benchmarks • It is desirable to have high values for • yield • A/FR • productivity • Since yield and A/FR are closely related, a yield vs A/FR benchmarking chart would not be useful. • Yield vs. productivity or A/FR vs. productivity would likely be useful benchmarking charts.

19. Defect Removal Strategies - part 1 • Focused inspections and reviews • high level design reviews - structural issues • detail level design reviews - logic correctness • code reviews - implementation issues • Do not address • system issues in DLD • design issues in code reviews

20. Defect Removal Strategies - part 2 • Do thorough reviews the first time and then trust them. • Do thorough unit tests • black box • white box • Do system tests once unit testing is done • integration • system • regression

21. Defect Prevention • Finding defects is expensive so it is better to avoid them in the first place • Defect prevention costs are only incurred once, but their savings impact every future project • For PSP, defect prevention actions include improved design methods and prototyping

22. Defect Prevention Strategies • Set priorities for defect types that are • found most frequently • the most troublesome • easily prevented • In setting priorities consider the defects most often found in integration and system testing • Incorporate your prevention actions in your process scripts, checklists, and forms

23. Defect Prevention Process • Follow an established schedule • Select 1 or 2 defect types for initial action • Track and evaluate the effectiveness of the defect prevention actions • Make needed adjustments and continue