Cmpe 589

1. Cmpe 589 Spring 2008

2. Software Quality Metrics • Product  product attributes • Size, complexity, design features, performance, quality level • Process  Used to improve development and maintenance processes • Effectiveness of defect removal, the response time of the fix process • Project  Describe project attributes/characteristics and execution • The number of developers, the staffing pattern, cost, schedule and productivity

3. Software Quality Engineering Seeks relationships among in-process metrics, project characteristics, and end-product quality (then use this to improve)

4. Product Quality Metrics • Mean time to failure (MTTF) • Defect density • Customer problems • Customer satisfaction

5. Product Quality Metrics • Measured by • the number of “bugs” (functional defects) • How long the software can run before it “crashes” • MTTF is used in safety critical systems • Measures the time between failures • Defect density is used in commercial apps • Defects relative to the software size

6. Defect vs. Failures • Human mistakes: resulting in incorrect software operation • Failure- software module no longer carries out intended function or performance level

7. Defect Density Metrics • Time frame – L.O.P – Life of Product • Four years (?) • LOC – Lines of Code • KLOC – Thousand lines of code • Time Frame/ LOC or KLOC

8. Lines of Code • Count only executable lines • Count executable lines plus data definitions • Count executable lines, data definitions, and comments • Count executable lines, data definitions, comments, and job control language • Count lines as physical lines on an input screen • Count lines as terminated by logical delimiters

9. Lines of Code • In the context of defect rate calculation • Productivity studies • The amount of LOC is negatively correlated with design efficiency • Enhancements and new versions • LOC count for the entire product and the changed code • Defect tracking

10. Maintenance Process • Answer: compute defect rates for new and old code • Change flagging- (comments) – ID number linked to specific requirement, version release number

11. Function Points • A collection of executable statements that performs a certain task together with declarations of the formal parameters and local variables manipulated by those statements • Originated by Albrecht at IBM in mid 70s

12. Function Points • Weighted total of 5 major components: • Number of external inputs (transaction types)X4 • Number of external outputs (report types)X5 • Number of logical internal files (the ones that user may concieve, not the physical files)X10 • Number of external interface files (files accessed by the apps but not maintained by it)X7 • Number of external inquiries (types of online inquiries supported) X4 • These are average weighting factors

13. Function Points • There are low and high weighting factors depending on the complexity assessment of the app.: • External input: low complexity,3; high complexity, 6 • External output: low complexity, 4; high complexity, 7 • Logical internal file:low complexity, 7; high complexity, 15 • External interface file: low complexity, 5; high complexity, 10 • External inquiry: low complexity, 3; high complexity, 6

14. Function Points • Complexity is defined as set of standards according to objective guidelines • e.g. For the external output: if the no of data element types is 20 or more and the no of file types referenced is 2 or more, than complexity is high. • Calculate Function Counts (FC): FC = ΣΣ w  x 5 3 ij ij i=1 j=1

15. Function Points • Scale from 0 to 5 to assess the impact of 14 general system characteristics: • Data communications • Distributed functions • Performance • Heavily used configuration • Transaction rate • Online data entry • End-user efficiency • Online update • Complex processing • Reusability • Installation ease • Operational ease • Multiple sites • Facilitation of change

16. Function Points • Sum up the scores and calculate Value Adjustment Factor (VAF) VAF = 0.65 + 0.01Σc¡ c¡ = the score for general system characteristic i. • The number of function points is obtained: FP = FC x VAF International Function Point User’s Group Standard (IFPUG, 1999) 14 i=1

17. Function Points • Issues related to the function point metric: • More research is needed on: meaning of FP and the derivation algorithm • Various other standards than IFPUG • Time consuming and expensive • Accurate calculation need certified FP specialists

18. Function Points: example • From Jones (2000). Software Assessments, Benchmarks, and Best Practices • The average no of software defects in US is approx. 5 per function point during the entire life cycle • Defect removal efficiency is calculated by the level of CMM • The estimated defect rates per function point: • Level 1: 0.75 • Level 2: 0.44 • Level 3: 0.27 • Level 4: 0.14 • Level 5: 0.05

19. Customer Satisfaction • Customer problems metric- problems using product • Problems per user month (PUM) = (number of valid defects/time period) • Compute this every month if you want to lower PUM • Ex. Number of installed licenses times the number sold per month to reduce PUM • Improve development process and reduce product defects • Reduce non-defects oriented problems- improve, support, usability, documentation, communication, training • Increase sales of installed licenses

20. Scopes of Three Quality Metrics Customer Satisfaction Defects Customer Problems