Defense Systems Acquisition

1. Defense Systems Acquisition LtCol Mike VarmetteCapital and Northeast Regional CampusProgram Management and Leadership Dept.July 2004

2. Why Worry About Risk? • Over the last decade downsizing, consolidation, shrinking budgets, increasing technological sophistication and shorter development cycles have lead us to examining adverse impacts on projects costs, schedule and performance. • The built in risk analysis we had with prescriptive standards (i.e. Mil Specs) is now gone as organizations and engineers move to performance standards. • Proactive Management vice Reactive Management

3. Management 1974 Peter F Drucker • The FORD vis SLOAN (GM) Story…”Management is needed not only because the job is too big for any one man to do himself, but because managing an enterprise is something essentially different from managing one’s own property” pg 384 • “Above all, disagreement is needed to stimulate the imagination. One may not need imagination to find the one right solution to a problem. But then this is of value only in mathematics. In all matter of true uncertainty such as the executive deals with – whether his sphere be political, economic, social, or military – one needs creative solutions which create a new situation. And this means that one needs imagination – a new and different way to perceiving and understanding.” pg 473 • “…the effective decision-maker compares effort and risk of action to risk of inaction. There is no formula for the right decision here. But the guidelines are so clear…act if on balance the benefits greatly outweigh cost and risk; and act or do not act; but do not “hedge” or compromise.” pg 476 • “As a specific discipline, management has its own basic problems, its own specific approaches, its own distinct concerns. A manger who understand the discipline of management will still be an effective – and may even be a first-rate – manager with no more than minimum competence in managerial skills and tools. A man who knows only the skills and techniques, without understanding the fundamentals of management, is not a manger ; he is, at best a technician. • Management is a practice rather than a science. In this, it is comparable to medicine, law and engineering. It is not knowledge but performance. Furthermore, it is not the application of common sense, or leadership, let alone financial manipulation. Its practice is based both onknowledge and on responsibility.”

4. RISK MANAGEMENT What is the issue… • PM Solutions Center for Business Practices (CBP) • Recent Study based on CMM and PMBOK • Risk Management lowest maturity in Programs • Risk Documentation the worst at 1.42 • Technical Req Definitions best at 2.58 • (www.cbponline.com) • DAU Risk Study showed…. • Strong on “knowledge” • Weak on application • Plenty of studies showing continued cost overruns Drucker…”knowledge and on responsibility”

5. A The Defense Acquisition Management Framework User Needs & Technology Opportunities • Process entry at Milestones A, B, or C • Entrance criteria met before entering phases • Evolutionary Acquisition or Single Step to Full Capability B C IOC FOC System Integration System Demonstration LRIP Full-Rate Prod & Deployment Sustainment Disposal Design Readiness Review FRP Decision Review Concept Decision System Development & Demonstration Operations & Support Concept Refinement Technology Development Production & Deployment Sustainment Pre-Systems Acquisition Systems Acquisition Capability Production Document (CPD) Initial Capabilities Document (ICD) Capability Development Document (CDD) Validated & approved by Validation Authority Relationship to Joint Capabilities Integration & Development System

6. A CURRENT STATUS *BASELINE -COST -SCHEDULE -PERFORMANCE EXECUTION STATUS PLANS *PROGRAM PLANS *EXIT CRITERIA RISK MANAGEMENT ASSESSMENT *COST *SCHEDULE *PERFORMANCE B C IOC FOC System Integration System Demonstration LRIP Full-Rate Prod & Deployment Sustainment Disposal Design Readiness Review FRP Decision Review Concept Decision System Development & Demonstration Operations & Support Concept Refinement Technology Development Production & Deployment Sustainment Pre-Systems Acquisition Systems Acquisition CURRENT STATUS *BASELINE -COST -SCHEDULE -PERFORMANCE EXECUTION STATUS PLANS *PROGRAM PLANS *EXIT CRITERIA RISK MANAGEMENT ASSESSMENT *COST *SCHEDULE *PERFORMANCE

7. DoD “old” PolicySelected References to RISK • DoDD 5000.1, The Defense Acquisition System • 4.5. Effective Management….tailor considering risk • DoD 5000.2-R, Mandatory Procedures for MDAPs/MAIS • Numerous references to RISK....management and mitigation • 1.2.4.2 Risk reduction in source selection criteria • 1.4.3.3.2 Cost Estimates include assessment of RISK • 2.3, 2.5, 2.9 Acquisition Strategy …reduce System-Level risk to acceptable levels…industry bear risks - 5.2.3.4.3…establish a risk management process • 7.4 Exit Criteria • DoDD 5000.4, OSD CAIG • The CAIG Chair report … include quantitative assessments of risk… • DoD 5000.4-M, Cost Analysis Guidance and Procedures • Para 1.E.1.2, … Subsystem Description address risk issues • Para 1.E.2.0, Risk..PM assess & plan to address/reduce

8. DoD “new” PolicyMore References to RISK • 30 Oct 2002 Deputy Sec Def Memo • 3.2 Tailoring…flexible approaches…risk, and complexity • 3.5 Reduced Cycle Time…Evol Acq preferred • 3.14 Knowledge-Based Acquisition… • Tech, Integration, and manufacturing risk reduced • 3.15 Systems Engineering approach • 3.20 Program Goals…implement management controls • APB with costs, schedule and performance • No discussion of risk... • 3.24 Cost Realism…cost risk and monitor (contract) • 3.25 Cost Sharing…undue risk is not imposed (contractor)

9. Risk Management Policy • Continually assess program risk • Develop Risk Management approaches prior to decision to proceed to next phase • Ensure Risk Management Encompasses • Identification • Tracking • Mitigation • Control • Ensure equitable and sensible allocation of risk between Government and Industry • Practice event-oriented management to emphasize prudent risk management.

10. Risk Management Procedures • Acquisition Strategy shall consider risk areas. • Establish Risk Management program to identify and control performance, cost and schedule risks. • Include Risk reduction measures in cost performance trade offs. • Monitor Risk throughout each acquisition phase to determine how risks have changed.

11. Evolutionary Acquisition Risk Management • Evolutionary acquisition is designed to get new military capabilities from the Lab to the warfighter as quickly as possible. • In the “old” process, we spoke of cost, schedule, performance. New process we speak of cost, schedule, performance and risk. • Acquisition programs are taking more risk and it is showing up in operational testing.

12. Risk Definition • Risk is a measure of inability to achieve overall program objectives within defined cost, schedule and technical constraints and has two components: • Likelihood of failing to achieve a particular outcome • Consequences of failing to achieve that outcome. For processes, risk is a measure of the difference between actual performance of a process and the best practice for performing that process *Risk Mgt Guide for DoD Acquisition, 4th Edition June 2003

14. DAU PM Tool Kit Feb 2002

15. Risk Planning • Develop an organized, comprehensive, and iterative approach • Identify adequate resources - People and $$ • Organize/Train Risk Management IPT Members • Develop Management Information System • Draft Risk Management Plan(Format - Risk Management Guide, Chap 5)

16. Risk Assessment • Identify: Risk Events • WBS elements analyzed against risk sources/areas • Analyze: Probability, Consequences • Determination of causes, impacts, sensitivity, and relationships • Rate/Prioritize: Risk events for handling Continuous Process Throughout Program Life Cycle

17. Risk AssessmentRisk Analysis • Refine description of risk events • Isolate causes of risk • Measure probability and impact of risk • What criteria should be used to determine High, Medium, and Low risk? • Probability/Impact based largely on judgement • Impact on quantitative terms, if possible • Tailored to the Product or Process

18. IDENTIFY

19. Risk Events:Identification • People • Users • Relationships • Decision Makers/Authorities • Organizations • Availability • Talent/Skill Level/education • Experience • Motivation/Morale

20. Risk EventsIdentification • Process Test facilities

21. Risk EventsIdentification • Technology • Change • New or Obsolete • Adoption/Use • Integration/Interfaces • Team (Government/Contractor) technology expertise • Security • Architecture • Scalability

22. Top CLAWS Program Risk Areas • Family of Systems • Performance Risk • Software & Algorithm Development & Integration • Performance Risk • Development Testing Approach & Schedule • Performance/Schedule Risk • Weight Budget & Platform Stability • Performance Risk

23. CLAWS Top Risks 1 High 12 Safety Reqt’s Exceeds Spec 12 0.9 • 28 Family of Systems 42 6 28 0.8 • • • 11 Safety of Crew in Cab 11 0.7 • 6 PRS #2 Availability Moderate Probability of Occurrence (Pf) 0.6 18 Reloader Redesign 18 32 0.5 • 32 Safety impacts Design • 22 45 15 0.4 • • • 15 High Failure Rates Low 0.3 42 DT Duration too short 0.2 45 Turret Servo Maturity 0.1 22 Insufficent Spares 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 Potential Severity of Consequence (Cf)

24. HIGH MODERATE HIGH HIGH PROBABILITY OF OCCURRENCE LOW MODERATE HIGH LOW LOW MODERATE LOW HIGH LOW SEVERITY OF CONSEQUENCES (IMPACT) Risk Rating Table

25. Risk Evaluation Classification/Ratings HPLI HPHI H I G H MED HIGH 3 1 4 2 L O W LOW MED LPLI LPHI HIGH LOW LiKELIHOOD IMPACT

27. Likelihood and Impact Criteria Definition The IPT needs to define the criteria for likelihood and impact for evaluating your project. • Likelihood • Very Likely (Example: “expect this risk greater that one in three chance to occur”) • Not Likely (Example:“expect this risk less than a one in three chance to occur”) • Impact(Criteria for Cost and/or schedule and/or performance and/or other programs) • Little Impact • Big Impact

28. RISK HANDLING STRATEGIES

29. Risk Handling (CAAT) • CONTROL • Reduce probability of event occurrence(P3I, Reuse S/W, Parallel Design) • AVOID • Use another path(Redesign, Eliminate Reqr, Change IOC, COTS) • ASSUME (Accept) • Make no changes(Trade Space = Cost, Schedule, & Performance; Risk Reserve) • TRANSFER • Reduce Impact(Warranties, FP Contracts, Insurance)

30. Risk Handling Strategy #1CONTROL: Reduce Likelihood • DEFINITION:Lowering the frequency of risk event occurrence. • EXAMPLE: Providing a redundant power supply for a computer system to increase overall system availability and reducing the number of downing events.

31. CONTROL: Reduce Likelihood: Tools and Techniques

32. Risk Response Strategy #2CONTROL: Reduce Impact • DEFINITION: A method of controlling the risk event by softening the effect or impact should the risk event occur. • EXAMPLE: An aircraft having an auxiliary hydraulic system in the event the primary system fails

33. Reduce Impact: Tools and Techniques

34. Risk Response Strategy #3Avoid • DEFINITION: Avoidance of the risk areas or sources that could possibly lead to the risk event. • EXAMPLE: Eliminate a requirement to build a subsystem because the technological risk is too high.

35. Avoid: Tools and Techniques

36. Risk Handling Strategy #4ASSUME • DEFINITION: Acknowledging a future risk event and accepting the potential consequences without any efforts to control it. • EXAMPLE:Only having “one deep” project team member and consciously not training a backup. ( Either too costly or not enough resources available.)

37. ASSUME:Tools and Techniques

38. Risk Handling Strategy #5Transfer • DEFINITION: Reduction of risk exposure by reallocation of risk from one part of the system to another or redistributing risk between the Government and the prime contractor or between Government agencies. (Part of the functional allocation process) • EXAMPLE: The Marine Corps decides to have the Army as lead service to develop a new tracked recon vehicle.

39. Transfer: Tools and Techniques

40. Quantification Measures - Strategies box • Strategies to handle risks • Measures (quantification) related to the risk • How does Strategy effect Measures • Costs • Schedule • Performance • Other Programs

41. Risk Monitoring & Reporting • Systematically track & evaluate performance of identified risk areas & events against established metrics • Periodic IPT meetings & Integrated Baseline Reviews (IBR) to discuss status • Defense Acquisition Executive Summary (DAES) & Selected Acquisition Reports (SAR) • Milestone, Decision, Progress Reviews • Continuous Risk re-assessment to reveal Unk/Unks, refine Known/Unknowns.

42. Conceptual Risk Management Reporting System RISK MANAGEMENT CONCEPT STANDARD REPORTS REQUEST OR OTHER CREATE REPORT SUBMIT DATA CONTRACTOR FOR ENTRY DATABASE RISK AD HOC MANAGEMENT FUNCTIONAL COORDINATOR REPORTS SYSTEM IPTs REQUEST REPORTS OR HISTORICAL INFORMATION (CONTROLLED ACCESS) DATA

43. Risk Control • TRACK AND EVALUATE RISK HANDLING AGAINST METRICS • TEST AND EVALUATION • EARNED VALUE • TECHNICAL PERFORMANCE MEASURES

44. WHEN TO REVIEW RISKS High risks- weekly agenda items for team meetings Medium risks – monthly reviews Low risks – each milestone, large program changes, or when the Risk Plan is redone

45. Risk Documentation • Important to Document • Lessons learned spread to other programs • Risk tracking • Many tools available such as Risk Radar • Easy to make up your own on on Excel, Access or Word

46. Risk Management: Deficiencies • Process weakly structured. • Process too subjective. • Risk likelihood overemphasized and impact underemphasized. • Risk plans unlinked to plans and milestones • Resources not assigned for risk mitigation. • Inadequate documentation.

47. Risk Management: Workarounds • When changes occur, review project risk plan. • Check risks along critical path. • Check high risks outside of critical path. • Check WBS. • Involve all appropriate IPT members. • Ensure program team has expertise and/or resources to mitigate new risks.

48. CONCLUSION • Risk Management techniques can be applied in almost all areas of project management. • Budgets, Schedules, Requirements, Contracts – all interrelated to Risk Management Plan • RM techniques must be continuously and iteratively applied. • This is a team not an individual effort!

50. Parking Lot Some questions • Does your program have/had Risk Management Plan ? • Contractor plan or PROGRAM plan ? • Are you following the plan? On all activities? • Do you have a Risk Board/Panel/Group ? • How often does it meet ? Does it have influence ? • How many risks are you tracking ? • Do you quantify your risks? • How integrated is your risk process with other tools • Do you quantify risk in your… • Cost/Budget Estimates … Schedule/Network? • Requirements/Technical requirements matrix • Do you force issues into your risk process? • What percentage of issues were foreseen?