Software Process and Project Metrics

1. 1 Software Process and Project Metrics

2. 2 Measurement is fundamental to any engineering discipline Measurement enables us to characterize, to evaluate, to predict, and to improve

3. 3 When you can measure what you are speaking about and express it in numbers, you know something about it; but when you cannot measure, when you cannot express it in numbers, your knowledge is of a meager and unsatisfactory kind: it may be the beginning of knowledge, but you have scarcely, in your thoughts, advanced to the stage of a science --- by Lord Kelvin

4. 4 Measurement & Metrics

5. 5 A Good Manager Measures

6. 6 Definitions Metric is a quantitative measure of the degree to which a system, component, or process possesses a given attribute -- by IEEE When a single data point has been collected, a measure has been established An indicator is a metric or combination of metrics that provide insight into the software process, a software project, or the product

7. 7 Metrics in the Process and Project Domains Many of the same metrics are used in both the process and project domain Determinants for software quality and organizational effectiveness

9. 9 Process Metrics Majority focus on quality achieved as a consequence of a repeatable or managed process Derived based on the outcomes that can be derived from the process Errors, work products delivered, � Derived by measuring the characteristics of specific software engineering tasks The effort or the time spent performing the umbrella activities

10. 10 There are �private and public� use for different types of process data Private metrics should be private to the individual and serve as an indicator for the individual only include defect rates (by individual), defect rates (by module), and errors found during development Conforms well with the personal software process (PSP)

11. 11 Public metrics generally assimilate information that originally was private to individuals and teams Project level defect rates, effort, calendar times, and related data

12. 12 Software metrics etiquette Use common sense and organizational sensitivity when interpreting metrics data Provide regular feedback to the individuals and teams who collect measures and metrics Don�t use metrics to appraise individuals Work with practitioners and teams to set clear goals and metrics that will be used to achieve them Never use metrics to threaten individuals or teams Metrics data that indicate a problem area should not be considered negative Don�t obsess on a single metric to the exclusion of other important metrics

13. 13 Statistical Software Process Improvement (SSPI) You can�t improve your approach to software engineering unless you understand where you�re strong and where your�re weak Use software failure analysis to collect information about all errors and defects encountered as an application is developed and used

14. 14 Project Metrics Used during estimating new projects and product quality Effort/time per SE task Errors uncovered per review hour Scheduled vs. actual milestone dates Changes (number) and their characteristics Distribution of effort on SE tasks

15. 15 Software Measurement Direct measures include cost and effort applied Line of code (LOC) produced, execution speed, memory size, and defects reported over some set period of time Indirect measures include functionality, quality, complexity, efficiency, reliability, maintainability, and other abilities

16. 16 Typical Size-Oriented Metrics errors per KLOC (thousand lines of code) defects per KLOC $ per LOC page of documentation per KLOC errors / person-month LOC per person-month $ / page of documentation

17. 17 Measures should be normalized Size-oriented metrics are not universally accepted as the best way to measure the process of software development

18. 18 Typical Function-Oriented Metrics Based on function points (FP) errors per FP (thousand lines of code) defects per FP $ per FP pages of documentation per FP FP per person-month

19. 19 Why FP Measures?

20. 20 Analyzing the Information Domain

21. 21 Computing Function Points

22. 22 Taking Complexity into Account

23. 23 The relationship between lines of code and function points depends upon the programming language and the quality of the design Figure in page 94 A historical baseline of information should be established before these metrics are employed for estimation

24. 24 Measuring Quality Correctness � the degree to which a program operates according to specification Defects per KLOC Maintainability�the degree to which a program is amenable to change Mean-time-to-change (MTTC) Spoilage � the cost to correct defects encountered after software has been released

25. 25 Integrity�the degree to which a program is impervious to outside attack Integrity = ? [ (1- threat) X (1 � security) ] Threat is the probability that an attack of a specific type will occur within a given time Security is the probability that the attack of a specific type will be repelled Usability�the degree to which a program is easy to use

26. 26 Defect Removal Efficiency

27. 27 Integrating Metrics Within The Software Process Realistically establishing a successful company-wide software metrics program is hard work But it is worth If we do not measure, there is no real way of determining whether we are improving If we are not improving, we are lost

28. 28 Metrics being collected can answer Which use requirements are most likely to change? Which components in this system are most error prone? How much testing should be planned for each component? How many errors (of specific types) can I expect when testing commences? �

29. 29 Establishing a Metric Baseline To be an effective aid in process improvement, a metric baseline must be established The process (Figure in page 101) Data collection ? Measures Metrics computation ? Metrics Metric evaluation ? Indicators

30. 30 Managing Variation The same process metrics will vary from project to project Statistical process control Control chart for determining whether changes and variation in metrics data are meaningful The moving range (mR) control chart The individual control chart

32. The procedure required to develop a mR control chart for determining the stability of the process Calculate the moving ranges Calculate the mean of the moving ranges Multiply the mean by 3.268. Plot this line as the upper control limit (UCL) If all the moving range values inside the URL, the process metrics is stable

34. 34 Metrics for Small Organizations Keep it simple Customize to meet local needs Be sure it adds value

35. 35 Homework 4.2, 4.3, 4.8, 4.11, 4.13