November 8, 2007

1. Conference Brief RISC-IQ: Risk Integrated w / Schedule & Cost – Intelligent Quantification Applications for Systems Engineers November 8, 2007 This document is confidential and is intended solely for the use and information of the client to whom it is addressed. International Council on System Engineering

2. The need for Risk Management to be Integrated with Systems Engineering is well understood • In a recent Department of Defense briefing titled “Technical Excellence through Systems Engineering”, it was determined that there was a lack of understanding in programs on systems engineering. One of the reasons sighted was: • Cost and schedule estimation and risk management processes inconsistently aligned with systems engineering processes • 26% of surveyed systems engineers listed “Integration with other program management” as a way to drive technical rigor back into programs. This included: • Linkage with acquisition strategy, IMP, IMS, logistics, testing, and risk management • As a result, a recommended strategy for driving technical rigor back into programs was to have risk management integrated, effective, and well resourced

3. So what? • The question isn’t “Should we do risk management?” • The question is, as a Systems Engineer: • What do I do with the risk program I have? • Is the risk program providing a good return for the expense? • What, if anything, are the risk program outputs telling me? • Can I use the risk program to communicate more effectively? • The rest of this discussion supposes a mature risk program exists: • Roles and Responsibilities are defined (i.e. a Risk Mgt plan exists) • Training has been provided • Resources have been provided (tools, staff) • Risk Management products are reviewed at regular intervals in formal reviews (Risk Review Boards)

4. RISC-IQ was developed to help extract actionable information from traditionally stove-piped data streams Risk Exposure? Impact Relationships? Goals Too Risky? Which Design? More Reserves? Major Drivers? Adequately Mitigated? RISC-IQ enables critical decision making

5. Baseline Product Development Economic Effectiveness Analysis Initial Risk Accounting Risk-WBS Mapping Mitigation Plan Development Mitigation Resource Analysis Program Risk Baseline Risk Analysis Schedule Factor Analysis Cost Factor Analysis Most Likely Cost Estimate Cost Estimating Mitigation Strategy Revision Economic Effectiveness Assessment Risk Impact Analysis Post-Mitigation Cost Estimate Integrated Master Schedule Schedule Development Pre-Mitigation Cost Estimate Pre-Mitigation Schedule Estimate Post-Mitigation Schedule Estimate RISC-IQ works by establishing a structured, repeatable process that integrates the potential impacts of risk on program baselines Risk Integrated with Schedule and Cost – Intelligent Quantification Objective: Compare program risk profile at mitigation completion to determine outstanding risk exposure. Objective: Assess completeness of baseline documentation and understand existing processes. Objective: Provide insight into where risks affect the program and uncover their true impacts.

6. Agenda Project / Program Initiation Design Cycle Selection Engineering Trades Defining Resources Measuring Execution Managing Groups of Projects

7. RISC-IQ uses explicit, quantified statements for risk probability and impact to determine risk scores VS • The right-hand set of tables: • Communicates the leadership’s sensitivity to risk • Reduces personal bias • Creates a foundation for quantitative analysis

8. Agenda Project / Program Initiation Design Cycle Selection Engineering Trades Defining Resources Measuring Execution Managing Groups of Projects

9. Waterfall Development Model & Derivatives Assists with estimates of overall cost and schedule estimates Maps risk mitigation activities to development phases Quantified risk score can be used to establish threshold for entering the next development phase Spiral Type Develop Model Enables capability vs. risk trade study for design features Economic analysis of risk help apportion development resources to highest need areas Tracking of risk profile useful in determining if the current spiral met its goals RISC-IQ can be used to help select and support multiple types of development cycle

10. Agenda Project / Program Initiation Design Cycle Selection Engineering Trades Defining Resources Measuring Execution Managing Groups of Projects

11. RISC-IQ is useful for evaluating and communicating the impacts of design selection on risk exposure • As part of design trade activities, RISC-IQ provides input to the engineering processes by analyzing design options based on risk profiles • The risk team helps determine the risk variability across the options and the dependency of any one risk to key features/designs within an option • RISC-IQ applies a “Dependency” framework for this activity. • Design Independent Risks- Risks that remain relatively constant when key aspects of the system are altered • Design Dependent Risks- Risks that vary significantly based on changes to the system design Design Dependent Risk Example Design Independent Risk Example Option 1 Option 2 Option 3 Option 1 Option 2 Option 3 Risk Score 11.23 5.48 55.75 Risk Score 32.50 32.00 33.50 Little to no change in the risk score as the design or option changes Significant change in the risk score as the design or option changes

12. RISC-IQ provides a simple analytical framework for quickly highlight what risks drive any given design option • Dependency analysis is performed for every risk to determine whether the risk applies to a given option and the strength of dependency by design parameter • Shown below is a sanitized example from a recent client engagement Results Sample

13. The RISC-IQ design dependency approach establishes a method to evaluating requirements and overall risk tolerance • Using the quantitative risk scores we can compare a single risk area across platforms, or the variability of a single risk within a given platform (Placing the options in order of complexity, i.e. performance or technology, allows the program to understand the ‘risk curve’ as the options become more aggressive or challenging in their profile.) Critical Critical High Risk Level High Risk Level Medium Medium Low Low Option 1 Option 2 Option 3 Option 4 Satellite C option Satellite D option Satellite B option Satellite A option Communications Risk Weight Risk

14. Once the analytical framework and quantitative scoring is established, risk can then be treated as a design variable • Risk can be treated as an independent variable and applied in a similar fashion to cost in CAIV methodologies; “Risk As An Independent Variable” (2) Performance Analysis working group determines relationship between the design variable and system performance (1) Risk curves are generated for designs or aspects of each design using dependency model (3) Utility Analysis working group performs effectiveness analysis to show relationship to performance metric Performance Sensitivity Risk Sensitivity Effectiveness Analysis Performance Metric Risk Level Effectiveness Technology Design Variable/Option Technology Design Variable/Option Performance Metric Effectiveness Variable Metric X X ƒ(Variable) = Metric Risk Variable

15. RISC-IQ then uses this analytical framework show when additional performance isn’t worth the additional risk • By combining these curves, it’s possible to identifying the point of diminishing risk tolerance for any design variable or option • Point of Diminishing Risk Tolerance- “The point at which the increase in additional design capability, or the selection of a proposed option, reduces effectiveness by increasing risk to the program.” Point of Diminishing Risk Tolerance Risk Tolerance Design Variable/Option Effectiveness = Risk

16. RISC-IQ takes the results of the design trades and evaluates them by source category and the work to satisfy the requirements • The following is an example from a commercial satellite project analysis A review by categories show holes in existing risk areas that should be investigated further. A review by WBS elements shows which areas of the program need additional risk identification. Critical Risk Categories with Few Risks Identified

17. Agenda Project / Program Initiation Design Cycle Selection Engineering Trades Defining Resources Measuring Execution Managing Groups of Projects

18. RISC-IQ uses a convention of aligning risks to the Work Breakdown Structure (WBS) to help define resource requirements • Aligning risks to the program WBS serves as the basis for allocating cost and schedule impacts throughout the program • During the risk analysis phase, WBS elements are correlated to each other and risks mapped to WBS elements Mapping provides an easily understood foundation that is repeatable, traceable, and defensible

19. 1.1.2.3 WBS 1 3 2 1 Baseline Estimate Risk-Adjusted Estimate 50% 50% $1,179,000 RISC-IQ uses this WBS mapping to show the impacts of risk on the original estimate • After mapping, any WBS element may have multiple risks that affect it • The combined affect of the risks on the cost element is related to the type of the risk, its likelihood and severity, and the cost of mitigation actions that could avert the risk • Using probabilistic models, the combined affect is determined and applied to the base cost estimate

20. RISC-IQ then applies the WBS mapping and pre- and post- mitigation scores to guide mitigation activity • The final determination of cost considers the remaining risk exposure AFTER mitigation, plus the mitigation investment required Mitigation Investment= $500,000 Represents the initial baseline cost distribution without any included risk effects $645,000 Represents the cost distribution with potential risk effects BEFORE any mitigation Represents the cost distribution AFTER successful mitigation, PLUS the required mitigation investment. As a result of a $500,000 investment in risk mitigation, the program avoided a potential increase of $645,000. This activity helps justify mitigation requests and can highlight when mitigation is no longer effective

21. The same process can be performed on any unit of measure. Risk Adjusted Schedule is particularly complex • Relationships between risks and schedule elements for critical path determination must account for: • Predecessor-Successor logic • Parent-Child • Parallel-Effort • Conditional Dependencies Risk Adjusted Delivery Likelihood Schedule Sensitivity By Task

22. Agenda Project / Program Initiation Design Cycle Selection Engineering Trades Defining Resources Measuring Execution Managing Groups of Projects

23. As a “snapshot” of project health, RISC-IQ can reveal whether or not the engineering activity is likely to deliver the expected results In the example below, a RISC-IQ analysis demonstrated a previously unknown risk exposure of $148M and potential project cost growth of 25% Incorporating RISC-IQ throughout a project’s lifecycle results in better initial planning and helps answer questions like: Are prototype or testing activities reducing risk as planned? Have risks been reduced sufficiently to pass entry / exit criteria for established technical reviews? Are new risks been uncovered faster than they are being closed? What are the impacts of “mid-stream” requirements changes? RISC-IQ can help determine if the engineering activity is proceeding as planned

24. Agenda Project / Program Initiation Design Cycle Selection Engineering Trades Defining Resources Measuring Execution Managing Groups of Projects

25. RISC-IQ practices for a single project are extensible to groups of projects or systems of systems the must interact to deliver capability… • By standardizing risk analysis across all multiple programs, we provide insight into high risk programs and overall trends Average Composite Risk Ranking by Program Critical High Average Risk Score Med Low

26. … and program-by-program risk exposure analysis provides comparative insights what programs can be improved Analysis will lead to a cumulative assessment of the total risk exposure, the investment allocated to reduce it, and the resultant investment benefit. $39,100 Initial Program Estimate Program Cost (in thousands) Revised Estimate Budget at Risk Mitigation Investment Reduced Cost Exposure

27. T0 T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11 T12 T13 T14 T15 T16 T17 T18 T19 T20 In a matrixed organization where Systems Engineering is a shared resource, using RISC-IQ to align risk-adjusted program schedules helps prioritize the timing of resources and risk reduction Contract Delivery Post-Mitigation Delivery Pre-Mitigation Delivery Program1 Contract Length Added Duration Total Risk Exposure Risk exposure is the amount of schedule slip reduced through successful risk mitigation The overlap of additional program duration requires allocation of program and/or enterprise resources Program2 Program3 Understanding most probable program delivery allows S&IS to: Program4 • Inform clients of most probable schedule • Plan for resources in advance • Address contract issues • Evaluate effectiveness of proposed mitigation • Understand when enterprise reserves will be needed ProgramN

28. In summary, risk management provides the Systems Engineer with quite a bit of “so what” Concept Development Design Planning & Execution Enterprise RISC-IQ can help established risk tolerance and incorporate risk as variable benefit trades RISC-IQ highlights what specific requirements or technologies are project drivers RISC-IQ provides unique insight into project cost and schedule realism, reserve requirements and health during execution RISC-IQ at the enterprise level highlights overall reserves needs, resource shortfalls due to slips, and supports benchmarking

29. For more information on how risk management can be applied to your specific challenges, please contact: Tweed Ross Colorado Springs, CO 719-322-5191 ross_tweed@bah.com Robert Makar San Diego, CA 619.725.6586 makarr@bah.com Michael Lopez Los Angeles, CA 310.291.2111 lopez_michael@bah.com