Module 1

1. Module 1 Welcome

2. Overview • Introductions • Administrative • Course Objectives • Course Schedule • Style of Course • Course Materials

3. Administrative • Breaks • Sign-In Sheet

4. Targeted Audience • Mix of project personnel with variable levels of experience in KSC development projects • Prerequisites: • Project management or systems engineering experience (at least one year) • Assumptions: • Prior knowledge of risk or risk management is not required

5. Course Objectives • Understand the concepts and principles of Continuous Risk Management and how to apply them • Develop basic risk management skills for each component of Continuous Risk Management • Be aware of key methods and tools • Understand how CRM could be tailored to a project • Be able to differentiate between Risks and Problems

6. Module 2 Introduction

7. Overview • Agency Risk Management (RM) Requirements • Risk Definitions • RM/Project Management Relationship • Risk Management Benefits • Continuous Risk Management (CRM) Process • CRM Process Application • Risk Management Planning/Documentation • Who Performs Continuous Risk Management?

8. Agency Risk Management Requirements • Risk Management Documentation • NPD 7120.4, Program/Project Management, describes the management systems for program/project formulation, implementation, and evaluation • NPG 7120.5, NASA Program and Project Management Processes and Requirements, dictates basic risk management requirements • NPG 8000.4, Risk Management Procedures and Guidelines, provides additional information for applying risk management to programs and projects as required by NPG 7120.5 • Procurement Notice (PN) 97-58, Risk Management

9. Agency Risk Management Requirements • Fundamental Risk Management Requirements • Program and project decisions shall be made on the basis of an orderly risk management effort • Risk management includes identification, assessment, mitigation, and disposition of risk throughout the project formulation, approval, implementation, and disposal phases • Project/Program Manager (PM) has the overall responsibility for the implementation of risk management, ensuring an integrated, coherent risk management approach throughout the project

10. Agency Risk Management Requirements • Fundamental Risk Management Requirements • Risk management planning will be developed during the project/program formulation phase, included in project/program plans, and executed during the implementation phase • Programs and projects will develop and maintain prioritized risk lists • Programs and projects must develop and clearly communicate “success criteria” to all levels of the program and project to define the scope of the effort and to guide risk decisions

11. Agency Risk Management Requirements • Fundamental Risk Management Requirements • Programs and projects must define, within the constraints imposed upon them (e.g., budget, schedule), what level of risk (i.e., “acceptable risk”) is reasonable for the program/ project to accept while still achieving mission success • Primary risks (i.e., risks having high probability and high impact/severity) must be classified, with acceptance rationale documented and concurred with by the Governing Program Management Council (GPMC)

12. Risk Definitions • Risk is the combination of the probability (qualitative or quantitative) that a program or project will experience an undesired event (cost overrun, schedule slip, safety mishap, security compromise) and the consequences (impact) of the undesired event, were it to occur. • NPG 8000.4

13. Risk involves the probability that an undesired event will occur. Qualitative or Quantitative Qualitative or Quantitative Risk Exposure = Probability x Impact Risk Definitions Risk involves the impact of the event should it occur.

14. Some Perspectives on Risk • Risk will always be present in programs and projects • Not all risk can be avoided • Management and stakeholders must be active participants in the mission risk acceptance process • Risks are different from problems

15. Goal of Risk Management • Achieving Mission Success • Provide program/project managers principles and practices for decision making • Search out, identify, and manage risk • Keep safety paramount • Deliver a quality product on time and within cost

16. Success Criteria Emphasis • Program/project teams must develop clear “Success Criteria” during the formulation phase • Success criteria must be clearly communicated to all levels of the program and project to define scope of the effort and to guide risk decisions • Success criteria need to be developed at system, subsystem, and component level to define trade space and support risk decisions • Success criteria will continue to evolve throughout the life cycle of the project

17. Schedule Performance Budget Risk Management People Quality Safety Configuration Management Risk Management/Project Management Relationship Project Management

18. Risk Management Benefits • Early identification of potential risks • Facilitates setting priorities • Increases chance of project success • Enables more efficient use of resources • Promote teamwork by involving personnel in all levels of the project • Information for tradeoffs is based on priorities and quantified assessments • Identify hidden risks

19. Dilbert Scott Adams Everybody Wants to Understand Risk

20. Continuous Risk Management Process • Continuous Risk Management (CRM) is a structured, formalized management practice with processes, methods, and tools for managing risks in a project • It provides a disciplined environment for proactive decision making to: • Assessment (continual) of what could go wrong (risks) • Determination of which risks are most important to deal with • Implementation of mitigation strategies to deal with those risks • Assured, measured effectiveness of the implemented mitigation strategies

21. Continuous Risk Management Process

22. CRM Process Components • Identify • Search for and locate risks before they become problems • Analyze • Convert risk data into useable information for determining priorities and making decisions • Plan • Translate risk information into planning decisions and mitigating actions (both present and future), and implement those actions

23. CRM Process Components • Track • Monitor risk indicators and mitigation actions • Control • Correct risk mitigation plans deviations and decide on future actions • Communicate and Document • Provide information to project on risk activities and current/future risks, and emerging risks

24. Relationship Among CRM Functions • Throughout the project life cycle, risk components evolve • Continuously • Concurrently • Iteratively

25. Risk Management Data Flow Statements of risk Project goals Context Resources and constraints Impact Probability Statements of risk Timeframe Individual Context Classification uncertainties Rank Classification Class 1 Class 2 Risk Risk Identify Analyze Plan Risk Risk Risk Class 3 Master list of risks Risk Risk List of risks Group/team Project Top uncertainties data N Statement of risk Decisions Context Status reports • replan Impact • close Probability • risks • invoke Timeframe • mitigation contingency Classification plans Resources • continue Rank tracking Plan Approach Statements of risk Context Statements of Risk Plan Track Impact Control Action plans Context Probability Impact Timeframe Probability Classification Timeframe Rank Classification Plan Approach Project Rank Risk & mitigation Project Status Plan Approach plan measure data data Status Control Decision

26. Testing Formulation Hardware Development Fabrication And beyond... Detailed Design Preliminary Design System Integration & Test Acquisition Hardware Requirements Analysis Control Flight Operations Identify System Requirements Analysis System Design Communicate & Document Track Analyze Software Requirements Analysis Disposal Plan Control Control Identify Identify Preliminary Design Communicate & Document Communicate & Document Track Track Analyze Analyze Plan Plan Detailed Design Code & Debug Integration Software Development Testing Continuous Risk Management Application

27. Formulation Implementation Approval Evaluation Reviews SRR FRR OAR NAR SAR DR CDR ORR PDR Phase A Preliminary Analysis Phase C Design Phase B Definitions Phase E Operations Phase D Development When Should CRM be done?

28. RM Planning/Documentation • Risk Management planning early in the project life cycle (i.e., formulation) is required per NPG 7120.5 (Section 4.3.2a); NPG 8000.4 • Options • Stand-Alone Risk Management Plan (Medium-to-Large Projects) • Risk Management Section in Project Plan (Smaller Projects)

29. Risk Management Plan Details • Purpose • Documents the risk management practice (processes, methods, and tools) to be used for a specific project • Contents • Introduction/Overview • Project organization, roles, responsibilities • Practice details (e.g., how risks are identified) • Risk management milestones (e.g., quarterly risk list reviews) • Risk information documentation (e.g., database) • De-scope options • Available Information • SE&T/YA-B, conjunction with SH&IA/QA-C, has established risk management and project plan templates, and can offer consulting and guidance during plan development

30. Relationship to Everyday Practice • Learning • Continuous Risk Management is similar to incorporating • any new habit • into your daily life.

31. Program/Project Management System Engineering Risk Management Board Safety & Mission Assurance Core Risk Management Team

32. Who Performs Continuous Risk Management? • Everyone!

33. Module 3 Identify Control Identify Track Communicate & Document Analyze Plan

34. Overview • Identification activities • Capturing statements of risk • Capturing the context of a risk • Identification methods and tools • Brainstorming • Questionnaires and checklists • Personal knowledge/experience • RM/S&MA analysis tools (FMEA, FTA, PRA) • Lessons Learned

35. Recording Data on the Risk Information Sheet • Risk Information Sheet • Fields to be Completed in Identification Phase: • ID • Date Identified • Risk statement • Origin • Risk Context

36. Capturing Statements of Risk • Purpose: • Arrive at a concise description of risk, which can be understood by everyone and acted upon • Description: • Involves considering and recording the condition that is causing concern for a potential loss to the project, followed by a brief description of the potential consequences of this condition

37. Components of a Risk Statement • Condition: A single phrase that identifies possible future problems, and describes current key circumstances and situations that are causing concern, doubt, anxiety, or uncertainty • Consequence: A single phrase or sentence that describes the key, negative outcome(s) of the current conditions there is a possibility that ; Given the Consequence will occur Condition Risk Statement

38. Elements of a Good Risk Statement • Consider these questions when looking at a risk statement: • Is it clear and concise? • Will most project members understand it? • Is there a clear condition or source of concern? • If a consequence is provided, is it clear? • Is there only ONE condition, followed by one (or more) consequence?

39. Example Risk Statements Good or bad risk statements? • Object Oriented Development (OOD)! • The staff will need time and training to learn object oriented development. • This is the first time that the software staff will use OOD; the staff may have a lower than expected productivity rate and schedules may slip because of the associated learning curve.

40. Capturing the Context of a Risk • Purpose: • Provide enough additional information about the risk to ensure that the original intent of the risk can be understood by other personnel, particularly after time has passed • Description: • Capture additional information regarding the circumstances, events, and interrelationships not described in the statement of risk

41. Contributing factors Risk source Interrelationships Condition ; Consequence Risk Statement Circumstances Context Context of a Risk Statement • An effective context captures the what, when, where, how, and why of the risk by describing the circumstances, contributing factors, and related issues (background and additional information that are NOT in the risk statement)

42. Elements of Good Context • Consider these questions when looking at the context • Can you identify which risk statement this context is associated with? • Is it clear what the source or cause of the risk is? • Is it clear what the impact might be? • Would you know who to assign the risk to for mitigation? Would they (the person responsible for risk mitigation) know what to do? • Would you be able to tell if the risk has gone away?

43. Example Context (#1) Risk Statement: • This is the first time that the software staff will use Object Oriented Development (OOD); the staff may have a lower-than-expected productivity rate and schedules may slip because of the associated learning curve. Risk context • It’s a typical NASA project – new concepts without training. • Is this an example of good or bad context?

44. Example Context (#2) Risk Statement • This is the first time that the software staff will use OOD; the staff may have a lower than expected productivity rate and schedules may slip because of the associated learning curve. Risk Context: • Object oriented development is a very different approach that requires special training. There will be a learning curve until the staff is up to speed. The time and resources must be built in for this or the schedule and budget will overrun.

45. Risks are Not Problems (1) • Risk: • A future event • A potential problem • Has a level of uncertainty (>0% and <100% chance of occurrence) • Problem • Is happening now • Must be dealt with immediately • No uncertainty (it’s occurring now)

46. Risks are Not Problems (2) • Risks are anticipated problems • Example: Delivery of Class S parts on schedule is questionable • A Problem is a Risk that has occurred • Example: Class S parts have not been delivered • Problems may create new risks • Change in design, increased testing • Schedule slip, screening cost

47. List of Risks Creative Energy Brainstorming Brainstorming Purpose: • Group method for generating ideas Description: • Participants verbally identify ideas as they think of them, thus providing the opportunity for participants to build upon or spring off of ideas presented by others

48. Risk Statement Identification Tools -- Taxonomy-Based Questionnaire (TBQ) • Taxonomy – The classification of something in an ordered system that indicates natural relationships; division into ordered groups or categories • TBQs are questionnaires organized according to the taxonomy of a particular body of knowledge • TBQs provide a structured approach for identifying risks associated with a project • CRM Guidebook (pp. 471-493)

49. Additional Risk Identification Methods • Failure Modes and Effects Analysis • Fault Tree Analysis • Probabilistic Risk Assessment (PRA) • Lessons Learned (http://llis.nasa.gov) • Various Other Checklists

50. Statement of Risk Risk Context Individual Uncertainties List of Risks Group/Team Uncertainties Project Data Risk Identification Data Flow Identify • Capture statement of risk • Capture context of risk