1 / 18

Projecting Quality

Projecting Quality. Previously, we used Defect Removal Effectiveness as a guidance to formulate a projection mechanism: MP : Problems Removed via Inspection(s) PTR: Problems Removed via Testing(s) (with the above 2 assumptions ---- then)

lysandra-bullock
Télécharger la présentation

Projecting Quality

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Projecting Quality • Previously, we used Defect Removal Effectiveness as a guidance to formulate a projection mechanism: • MP : Problems Removed via Inspection(s) • PTR: Problems Removed via Testing(s) (with the above 2 assumptions ---- then) • Q : Field Problems = PTR2/ (MP-PTR) • Are there other models we can use ?

  2. Projecting Field Problems • 2 Basic Models : • Static Model : • # of Field Problem or Q = f ( x1, x2, - - - - -, xn) where x’s are a set of parameters such as size of the software, skills of the people, development schedule, etc. • The coefficients of x’s used in the computation come from previous, similar projects. • The computation of # of Field Problems for the project is assumed to be one from a population of similar projects.

  3. Projecting Field Problems • Dynamic Model : • Estimate is based on multiple measurements taken from the project of interest, not from other projects, but from the current project • It is more specific to the project or product. • It is often expressed in terms of time. • We will use one such model to simulate the complete product life cycle.

  4. Weibull Distribution family • Has been used by other disciplines for reliability analysis and projecting the defect as a function of time. • The probability density function : f(t) = (m/t) * ((t/c)**m) * (e ** -((t/c)**m) ) = (m/t) (t/c)m e (–t/c) **m • m is a parameter that shapes the curve • t is the time • c is a parameter which is a function of t when the curve reaches maximum point. • The cumulative distribution function : (integrate the above) F(t) = 1 – (e** -((t/c)**m) )

  5. Weibull Curves (with differing m’s) f(t) = (m/t) * ((t/c)**m) * ( e** (-(t/c)**m) ) m= 8 problems found m= 2 m= 4 m= 1 t: some unit of time

  6. Rayleigh Curve • A special case of Weibull curve where m=2 is called the Rayleigh curve f(t) = (2/t) * ((t/c)**2) * ( e** (-(t/c)**2) ) At tmax , the area under the curve is ~39.3% problems found m= 2 tmax t: some unit of time

  7. Rayleigh Curve • The area under the f(t) curve (normalized to a total of 1) is the (% of)accumulative total problems found through some time. The integral of f(t) is the cumulative distribution function: F(t) = 1 - (e** (-(t/c)**2) ) total problems found t : time units

  8. Applying Rayleigh Curve concept to Discrete Measurements • Let’s map software defect removal activities into the Rayleigh Curve: (Some projects have found the mapping to hold) Io = requirement inspection I1 = design inspection I2 = code inspection UT= unit testing CT = component testing ST = system testing FP = field problems problems found I I CT ST FP or Q I UT 0 1 2 t : product life phases

  9. Projection of Field Problems with Rayleigh Curve • If your project appears to show a Rayleigh curve through the first couple of inspections • Continue to track against the curve • See if the peak point is close to UT phase. Assign that phase or time as the point where f(t) reaches the max point. • Use that max time to compute the value of “c”for the Rayleigh curve for your product: c= t * 2 (see next chart) • f(t) = (2/t) * ((t/c)**2) * (e** (-(t/c)**2)) max

  10. Solving for the “c” parameter • A Way to solve for the parameter, c, in Rayleigh curve • Take the first derivative of f(t) and set it to 0 to find the time at which f(t) peaks. • d[f(t)] = d[(2/t) * ((t/c)**2) * (e**(-(t/c)**2)) ] • = d[ (2t/(c**2)) * (e**(-(t/c)**2)) ] • = (2t/(c**2) ) d[e**(-(t/c)**2)] + d[2t/(c**2)](e**(-(t/c)**2)) • = (2t/(c**2)) [(e**(-(t/c)**2)) d[-(t/c)**2] + {second part} • = (2t/(c**2))[(e**(-(t/c)**2)) * (-2t/c**2)] + {second part} • = ((-4t**2)/(c**4)) (e**(-(t/c)**2)) + {second part} • = {first part} + (2/(c**2)) (e**(-(t/c)**2) • = (e**(-(t/c)**2)) *[( (-4t**2)/c**4)) + (2/c**2)] = 0 • (-4t**2)/(c**4) +(2/c**2) = 0 • (-4t**2)/(c**4) = - (2/c**2) • -(2t**2) = - (c**2) • t**2 = (c**2)/2 • t = c / (sqrt(2)) look for ‘max’

  11. Try an Example with Rayleigh Curve • Assume the following information (collected data) • I0 : at time 1 with 9 problems found • I1 : at time 2 with 12 problems found • I2 : at time 3 with 17 problems found • UT : at time 4 with 10 problems found • CT : at time 5 with 6 problems found • ST : at time 6 with 3 problems found • FP : problems in remaining time (field problems) • tmax is approximated at time unit 3, or I2, in this case = total of 57 problems

  12. Example Continued • Using the previous equation : tmax = c /sqrt(2) • c = tmax * sqrt(2) = 3 * 1.41 = 4.23 • Use the accumulative F(t)= 1 – e**(-(t/c)**2) • F(6) = 1 – e**(-(6/4.23)**2) = 1 – e**(-1.99) = 1- (.135) = .865 • Since 1 is the total area under the curve, the remaining area under the curve is .135. • Also since .865 is approximately equal to the total problems found up to end of system test, which is 57 problems, then the total life cycle is estimated at 66 problems. (57/.865 = 66) • Thus the remaining problems in the field is estimated at 66-57 = 9 problems. • FP (field problems) is estimated at 9 problems

  13. Compare with the Previous Estimation Method • Consider the previous estimation of Field Problems Q = PTR/(μ-1) = PTR / [(MP/PTR) -1] = PTR2/(MP-PTR) • Substituting the numbers : • Q = 192/(38 -19) • Q = 361/19 = 19 problems • Note the difference between the 2 estimates with the same assumed problems found rates : • Q with Rayleigh Curve approach estimated at 9 problems • Q with the PTR2/(MP-PTR) estimated at 19 problems. (Remember the “earlier” estimate assumed MP/TD = PTR/(TD-MP) which in this case 38/66 does NOT equal 19/28)

  14. Estimates • Remember that these are still estimates and not all the conditions are known, yet. • Two assumptions are used in using the Rayleigh curve to model quality of software: • The defect rate observed during development activities is highly correlated to the defect rate in the field. • Given the same error injection rate, then the more defects discovered early in the development cycle, the fewer will remain in the system at later stages.

  15. Correlating Development Phase I0-inspection, Defect Rate with Field Defect rate Rank field defect rate by module Rank I0 by module (Diff)2 Diff -3 -18 -22 -5 . . . +13 9 324 484 25 . . . 169 4 20 25 9 . . . 52 1 2 3 4 . . . 65 Spearman’s Rank Order Correlation coefficient = 1 – {[6* Σ(diff)2] / [n *(n2-1)]} Using this technique, book author found low correlation for I0, I1 to field defects and higher correlation of I2, CT, and ST to field defects

  16. More “thoughts” illustrated with Rayleigh curve Using the same process of error removals for a set of modules, we have been noticing that if we find lots of bugs in a module, X, relative to other modules, then that same module, X, tends to be more error prone (have high # of bugs) in the field also. [ A uniform upward shift effect - no shape change] # of Defects found More error prone module A “typical” module product release day Time

  17. More “thoughts” illustrated with Rayleigh Curve We have also justified the concept of finding more bugs prior to release is a good scenario because less will be in the product after release. So if more bugs are found in a module, X, relative to other modules prior to release should be a good sign. How does that relate to the comments on a module with lots of bugs found prior to release tends to be the same module that is error prone in the field? [ A change in shape of the curve ---due to modified process ] Modified or improved process and the number of problems removed shifted to the front, resulting in less problems detected after release. Original process and the number of problem found Product release

  18. Reliability and Predictive Validity of Models • Reliability of the Model refers to the confidence we have of the model in terms of its estimates. • Dos the output from the model vary greatly from one input set to another input set? • Predictive Validity of the Model refers to the accuracy of the model estimates, especially compared to empirical data • The author found that using Rayleigh curve model across different systems such as AS/400, S/38/ and S/36 data showed that the field problems discovery rate is consistently underestimated. The model had to be adjusted to set m= 1.8 for the IBM mid-range systems.

More Related