1 / 26

Some Topics In Measurement/Quantitative Analysis

Some Topics In Measurement/Quantitative Analysis. John E. Gaffney, Jr., PE gaffney123@verizon.net 301-509-8552 November 3, 2010. Topics. Some Topics In Measurement/Quantitative Analysis: Function Points Risk and Cost Estimation, COSYSMOR Defect Analysis, Software Reliability

paul
Télécharger la présentation

Some Topics In Measurement/Quantitative Analysis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Some Topics In Measurement/Quantitative Analysis John E. Gaffney, Jr., PE gaffney123@verizon.net 301-509-8552 November 3, 2010

  2. Topics • Some Topics In Measurement/Quantitative Analysis: • Function Points • Risk and Cost Estimation, COSYSMOR • Defect Analysis, Software Reliability • Some Observations

  3. Measurement/Quantitative Management • Measurement is all about supporting good management: • Knowing where you want to go; having the resources in place to get there; determining whether you are there or are likely to get there; and taking action if you are diverging from your goals • Quantitative Management is a ”closed-loop” process: 1.Establish goals, 2.Compare actuals with goals, 3.Take action if appropriate (e.g. a metric out of expected range; don’t act if O.K.) • The closed-loop (feedback) approach tends to compensate for “noise” in the system, analogous to feedback in control systems (electrical eng.) • Quantitatively characterize process, product, performance, personnel, tools/methodology

  4. Function Points: Some Observations • Invented by Allen J. Albrecht, IBM, in the1970’s. • Developed to capture “..user/customer requirements in a way that is more easily understood by the user/customer than SLOC.”* • Originally used to estimate development costs for a particular IBM organization for business applications; absorbed both size and productivity • Later, in more broadly-based usage, FP’s used as the metric for application size; then development effort was based on the “universal” size measure and the particular productivity • The usage of FP’s very quickly spread • For many, the use of FP’s in lieu of SLOC became virtually a religious matter • There are many variants of function points, e.g., feature points (Capers Jones), simplified function points (Gaffney) • Some related measures are: story points and Halstead’s “unique I/O count” • Function points may not be particularly well suited to use in highly calculation-intensive, “engineering” type software • SLOC count and function point count/value typically highly correlated * Albrecht,Gaffney, IEEE,TSE,Nov. ‘83

  5. From Albrecht/Gaffney Data

  6. From Albrecht/Gaffney Data

  7. From Albrecht/Gaffney Data

  8. From Albrecht/Gaffney Data

  9. Defect Management Process Defect management is best executed as a ”closed-loop” process On many projects it is not, however, and defect content, etc. are treated as “free variables,” with no goals established for their values Defect management should be viewed as just as much a part of management as are cost and schedule management Defect detection, analysis (including root cause analysis), correction, and avoidance have cost and schedule implications We need to know: where we are going (goal); measure progress against goals, determine whether we are likely to achieve each goal or already have done so (measure and compare); take corrective action as necessary A good measurement program is key to asuccessful Defect Management Process Active Defect Management provides potential early indication (headlight metrics) of success and/or of problems Software Reliability estimation: based on estimates of latent (delivered, relevant) defects and prospective rate of discovery

  10. Defect Tools Overview Purpose of the tools: fit defect discovery data to a curve to make predictions about discovery later in the development/testing process from data obtained earlier; track progress against goals, early course correction as required - Weibull curves: e.g. Rayleigh, exponential The tools are a key to implementing the software defect management process Two types of tools: activity-based and time-based Provide “headlight” metrics; e.g., can indicate defect discovery and content objectives are not likely to be realized; can provide better mid-course cost/schedule predictions Activity-based tools fit data obtained from development activity verification, e.g., code inspection, and from testing A key value add: provide early prediction of testing results and latent defect content before testing has started and during testing; this can help minimize rework costs Tools (Gaffney) evolution history: STEER I (IBM,1985; key point, made an activity-based tool; before then, time-based fits only, to JEG’s knowledge); SWEEP (Software Productivity Consortium, 1988 et seq.); STEER II Lockheed Martin, 1997 et seq.); DEFT ( Post-Lockheed 2010)

  11. Some Aspects of Uncertainty Management and Risk Modeling • Better management of uncertainty starts with recognizing its existence and in estimating cost/schedule/quality and other “risks” • Use the risk information to make better management decisions during both business acquisition and program execution • Serve as a basis for discussion of alternatives within the project as well as with the customer • A key aspect of “affordability” • Recognize that the actual project outcomes, e.g, size, cost, duration/schedule, quality, are uncertain as are many of the key determining parameters, e.g., productivity, size • Thus, the risk of not achieving desired objectives should be quantified to the degree possible. • “Risk” where smaller values are desirable, e.g., development effort, as used in COSYSMOR: Risk=Prob [Actual Effort Will Be >Value] Confidence=100%-Risk • “Risk” where larger values are desirable, MTBF and reliability: Risk=Prob [Actual Reliability Will Be <Desired Value] Confidence=100%-Risk

  12. A COSYSMOR PLOT: Person Hours Risk (Larger values of effort are less desirable) Target

  13. A COSYSMOR PLOT: Person Hours Overrun Risk Tail of the Previous Plot (Larger values of effort overrun are less desirable)

  14. Reliability: Prob of no failures during some time interval, starting at some point in time after system/software/item goes into service and the clock starts (Smaller values are less desirable)

  15. Original COSYSMOR Parameter Inputs Low, Likely and High values for each parameter; defines probability distribution using Keefer and Bodily (1983) non-parametric method

  16. Some Observations-1 • Expect your work life to offer you alternatives (and thus real possibilities for personal and professional growth) that you cannot plan for • Your education is never complete; look for opportunities to learn (formal and informal) • Don’t be afraid to challenge “accepted wisdom” if you believe that you are correct • Perhaps, no one else has done so or others are afraid to do so • Ask questions of “ancient worthies;” just because the boss or someone else has more experience or seemingly greater credentials, doesn’t necessarily make him or her correct (But, please do be polite !) • Ask questions of yourself, of your assumptions and conclusions

  17. Some Observations-2 Determine the use for the metrics to be collected/developed; what information are they going to provide/what questions are they going to answer or help to answer; who wants the information and what decisions are they going to make using that information Some keys to getting good metrics: use a development process that has well-defined activities, artifacts produced by each, well-defined entry/exit criteria; good definitions for the metrics; good collection and analysis processes; training of all personnel involved; strong commitment by both management and team members; recognition by project team members that the metrics are used to help manage the project and to make a product that has predictable attributes using a process that also has predictable attributes Observation 1: Lack of good historical and current data on which to characterize processes, products, performance and thence to support bases of estimates Observation 2: Mathematics/statistics skills are often found in project personnel at a lesser than desirable level; can lead to not knowing what to expect/over-confidence/no confidence

  18. Backup

  19. Additional Functions Provided By COSYSMOR COSYSMOR provides four major additional functions beyond those provided by Academic COSYSMO: • Estimation of Cost/Effort and Schedule Uncertainties/Risk and Confidence: Provides quantification of the impacts of uncertainties in the values of key model parameter values. Provides multiple cost and schedule values with associated probabilities. Risk=Prob [Actual Effort Will Be >Estimated Effort] Confidence=100%-Risk • Representation of Multiple Types of Size Drivers: Provides for entering counts of: new, modified, reused, and deleted types for each of the four size driver categories. • Labor Scheduling: Provides the spread of systems engineering labor for the five systems engineering activities and across four the development phases (time). • Labor Allocation: Provides for the user to select the percentage allocations of the twenty activity/phase pairs or effort elements.

  20. Affordability “Affordability” is a measure of a system’s effectiveness “Affordability” means that a given set of needs (performance requirements) can be met within stated cost and schedule constraints. “Affordability” can also be defined as the probability (confidence) of achieving a stated set of needs at a stated cost and schedule (effort). The associated “risk” is determined (estimated) on the basis of the capability of the organization to meet this set of needs. “Risk” equals 100% minus “Confidence” From presentation by Cole, Gaffney and Schimmoller at the 2009 PSM Conference 25 25

  21. Keefer and Bodily Three-Point Approximation To A ContinuousProbability Distribution* 0.05 Fractile↔0.185 Prob 0.50 Fractile↔0.630 Prob 0.95 Fractile↔0.185 Prob *D.L. Keefer and S.E. Bodily, 3-Point Approximations For Continuous Random Variables, Management Science, 2995), 1983, 595-609

More Related