1 / 26

Risk Analysis Estimating Uncertainty and what this has to do with the price of milk in McLean. The Society of Cost E

Slide 1 of 26. Agenda. The ProblemVariability

colby
Télécharger la présentation

Risk Analysis Estimating Uncertainty and what this has to do with the price of milk in McLean. The Society of Cost E

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

    1. Risk Analysis & Estimating Uncertainty and what this has to do with the price of milk in McLean. The Society of Cost Estimating & Analysis (SCEA) Greater Washington DC Chapter Phil Beenhouwer May 17, 2006

    2. Slide 1 of 26 Agenda The Problem Variability & Standard Deviation Briefing Goals An Uncertainty Primer Definitions Distributions and Simulation Excel Prowess Benefits, Headlines, Other Disciplines

    3. Slide 2 of 26 My Premise

    4. Slide 3 of 26 The Problem

    5. Slide 4 of 26 Variability and Standard Deviation When someone says itll cost you $100 The average American carries $9,000 in credit card debt.1 In reality, most Americans owe nothing to credit card companies. Most households that carry balances owe $2,000 or less. Only about 1 in 20 American households owes $8,000 or more on credit cards.

    6. Slide 5 of 26 Briefing Goals The threshold is for you to think in terms of risk, uncertainty, and variability

    7. Slide 6 of 26 An Uncertainty Primer (Slide 1 of 4) SDLC Phase Concept and Business Case Initiation and Authorization Project Definition System Design Construction Acceptance Operational Readiness Action Resolve the lack of milk at home Spousal approval/funding Buy a gallon of milk Find merchant along route home Drive to store and purchase milk Did you have enough money? Get a clean glass!

    8. Slide 7 of 26 Gallon of Milk Data Collection Convenience Store: $3.49 Warehouse Club #1: $2.59 Warehouse Club #2: $2.69 Grocery Store #1: $2.69 Grocery Store #2: $3.19 Grocery Store #3: $2.89 Grocery Store #4: $2.79 Specialty Store #1: $3.09 Specialty Store #2: $2.99 Specialty Store #3: $3.29 An Uncertainty Primer (Slide 2 of 4)

    9. Slide 8 of 26 An Uncertainty Primer (Slide 3 of 4) So, if you budgeted $3.00 for milk (the mean), you can go to Warehouse Club #1: $2.59 Warehouse Club #2: $2.69 Grocery Store #3: $2.89 Grocery Store #1: $2.69 Specialty Store #2: $2.99 Grocery Store #4: $2.79 But you cant go to Convenience Store: $3.49 Specialty Store #3: $3.29 Grocery Store #2: $3.19 Specialty Store #1: $3.09

    10. Slide 9 of 26 An Uncertainty Primer (Slide 4 of 4) What can you afford to buy at these stores? Is a half-gallon acceptable? Will you even leave with milk? How will Operational Readiness go when you get home? Can you use legacy orange juice in tomorrows cereal instead? Will you need to reduce the number of cereal users? Will you need to cut all cereal training from the budget? Will there be a GAO report on your pillow in the morning? Sure, you say, if I end-up at the Convenience Store with only $3.21, I can find the 28 cents I need from under the drivers seat. But what if these werent dollars, but billions of dollars? Could you find $28M under the Program Managers seat?

    11. Slide 10 of 26 Program Risk versus Estimating Uncertainty Total Risk = RiskProgram + (+/- UncertaintyEstimation)1 For the purposes of quantitative risk analysis, I have defined: Program Risk the probability and severity of loss linked to hazards2 (e.g., software development cannot begin if the environment is not ready, system testing might fail, etc.) Estimating Uncertainty -- the estimated amount or percentage by which an observed or calculated value may differ from the true value3 (e.g., the number of users could be 25% less, the COTS license cost could be $1,000 more, training could take one week longer, etc.) Should also consider benefits and schedule, in addition to cost

    12. Slide 11 of 26 Definitions Simulation: any analytical method meant to imitate a real-life system.1 Monte Carlo simulation: a simulation that randomly generates values for uncertain variables over and over to simulate a model.1 xth percentile: the percentage at which x% of all outputs are at, or below, the associated cost value i.e., the 80th percentile in the gallon of milk example means that 80% of the values are at, or below, $3.21. Conversely, 20% of the values exceed $3.21.

    13. Slide 12 of 26 Description of Monte Carlo Tools Crystal Ball applications transform your spreadsheets into dynamic models that solve almost any problem involving uncertainty, variability and risk.1

    14. Slide 13 of 26 Triangular Distributions

    15. Slide 14 of 26 Triangular Distributions

    16. Slide 15 of 26 Monte Carlo Simulation

    17. Slide 16 of 26 Monte Carlo Output

    18. Slide 17 of 26 Other Common Distributions

    19. Slide 18 of 26 With a little Excel prowess

    20. Slide 19 of 26 With a little Excel prowess

    21. Slide 20 of 26 Benefits to Quantifying Risk/Uncertainty & Using Monte Carlo Identify and apply risk/uncertainty within a model where it really exists (I.e., risk/uncertainty does not really exist +/- 10% around software integration!) Sensitivity analysis Risk-adjusted estimates are included in the individual items of the model instead of as a bottom-line tax Makes it harder for decision-makers to remove the risk line-item

    22. Slide 21 of 26 Headlines GAO Testimony: CAPITOL VISITOR CENTER -- Results of Risk-based Analysis of Schedule and Cost, GAO-06-440T, February 15, 2006 GAO Report: INFORMATION TECHNOLOGY -- Agencies Need to Improve the Accuracy and Reliability of Investment Information, GAO-06-250, January 2006 GAO Report: DEFENSE MANAGEMENT -- Additional Actions Needed to Enhance DODs Risk-Based Approach for Making Resource Decisions, GAO-06-13, November 2005

    23. Slide 22 of 26 Relationship to Other Disciplines Budgeting / Investment Planning Provides risk-adjusted requirements of the E300 and other planning activities Acquisition Cost, Schedule, and Performance are part of defining an acquisition strategy Contracting Provides a cost basis for negotiation Program Management Provides insight into Program risks; helps prioritize mitigations Its just good Program Management !! Earned Value/Baseline Management Provides an input to the management reserve level More objective inputs to the EVMS than the typical Integrator provides Quantification of risks and uncertainties will result in less re-baselines Engineering Offers an approach to incorporate uncertain aspects of the engineering design

    24. Slide 23 of 26 MS Project Monte Carlo Analysis

    25. Slide 24 of 26 Conclusion Think and speak in terms of risk and uncertainty (and then apply it) Collect uncertainty inputs when you gather data Use Monte Carlo applications (e.g., Crystal Ball, @Risk, etc.) These are relatively inexpensive compared to other applications in a costers toolkit There is even a $15 application that we are currently investigating (Excel Business Solutions XL Modeling: www.excel-modeling.com/index_007.htm) Include Program Risk and Estimating Uncertainty in cost, benefit, and schedule analyses

    26. Slide 25 of 26 And if all else fails.., re-define the word outlier

    27. Slide 26 of 26 Questions / Comments

More Related