LESSON 3 Quality Assurance Risk Assessment

1. LESSON 3Quality Assurance Risk Assessment

2. Definitions • Risk Consequence (aka Risk Impact) - is driven by the requirements of the contract. It represents the consequence of an uncertain event or condition occurring. Risk consequence cannot be influenced by GCQA surveillance. So, unless the customer changes the requirement, the QAS cannot ever change the risk consequence. However, the scope of the surveillance must be based on the risk consequence. • Risk Likelihood - is based on the supplier’s performance. It is perfectly acceptable to have a high-risk consequence and low-risk likelihood based on the supplier’s performance.

3. Lesson Introduction Given a contract technical data package, you will be able to complete a Quality Assurance (QA) risk assessment using the Risk Profile and Plan Tool.

4. Lesson Objectives Upon completion of this lesson, you should be able to: • Explain the importance of QA risk assessment to Government Contract Quality Assurance (GCQA) surveillance planning. • Outline the QA risk assessment process. • Initiate a Risk Profile using the Risk Profile and Plan Tool. • Develop a Facility Process List within the Risk Profile and Plan Tool. • Assess the consequence of identified risks using the predefined conditions listed in the Risk Statement Generator of the Risk Profile and Plan Tool. • Write a Risk Statement using the Risk Statement Generator in the Risk Profile and Plan Tool. • Assess Performance Factors for all suppliers and those with higher-level requirements.

5. Lesson Objectives (cont.) Upon completion of this lesson, you should be able to: • Identify potential causes of an identified risk using the Facility Process List and the Performance Factors Assessment section of the Risk Profile and Plan Tool. • Assess the likelihood that an identified risk cause will occur using the Likelihood Table in the Risk Profile and Plan Tool. • Identify the Data Collection and Analysis (DC&A) tab, to include the data to collect, data source, and scheduling data analysis. • Examine risk assessment results for use in GCQA surveillance planning.

6. Lesson Topics During this lesson, you will complete the following topics: • Importance of QA Risk Assessment • The QA Risk Assessment Process • Initiating a Risk Profile • Developing a Facility Process List • Risk Consequence • Writing Risk Statements • Identifying Potential Causes of Risks • Assessing Risk Likelihood • Developing a DC&A Plan • Examining Risk Assessment Results

7. WIIFM? This lesson is important because as a QAS, you will be required to perform the risk assessment process to plan/update GCQA surveillance.

8. Importance of QA Risk Assessment Lesson Topics: Importance of QA Risk Assessment The QA Risk Assessment Process Initiating a Risk Profile Developing a Facility Process List Risk Consequence Writing Risk Statements Identifying Potential Causes of Risks Assessing Risk Likelihood Developing a Data Collection & Analysis Plan Examining Risk Assessment Results

9. Topic 1: Importance of QA Risk Assessment Risk Assessment is the process used to identify, assess, and document risks to quality and technical performance of contracts that require GCQA surveillance.

10. Why Risk Assessment Risk Assessment: • Provides an effective means of determining the appropriate type and amount of GCQA. • Helps sort through all of the contractual requirements and identify risk causes for GCQA surveillance planning. • Captures the risks, the consequences of the risks, the potential causes, and the likelihood of the cause occurring through the use of the Defense Contract Management Agency (DCMA) Risk Profile and Plan.

11. The QA Risk Assessment Process Lesson Topics: Importance of QA Risk Assessment The QA Risk Assessment Process Initiating a Risk Profile Developing a Facility Process List Risk Consequence Writing Risk Statements Identifying Potential Causes of Risks Assessing Risk Likelihood Developing a Data Collection & Analysis Plan Examining Risk Assessment Results

12. Topic 2: The QA Risk Assessment Process • Risk assessment concept • Risk statement • Eight risk assessment process steps • Risk Profile and Plan Tool

13. QA Risk Assessment Concept

14. Risk Statements Risk Statement Cause Assessment What process, product, or system failures would cause the risk statement to occur? Likelihood Assessment What is the likelihood that the cause or potential causes will occur? Consequence Assessment What would happen if the risk statement were to occur?

15. Risk Assessment Process Steps Steps 1: Initiate Risk Profile 2: Develop/Update Facility Process List 3: Perform Risk Consequence Assessment 4: Generate Risk Statement(s) 5: Conduct Performance Factors Assessment 6: Identify Risk Cause(s) 7: Determine Likelihood 8: Complete Risk Profile

16. Question and Answer What is the first step in the Risk Assessment Process? • Develop Performance Factors for All Suppliers • Determine likelihood of risk cause • Identify risk cause • Initiate Risk Profile

17. Risk Profile & Plan Tool Spreadsheet Overview Individual sections of the Risk Profile and Plan Tool

18. Table of Contents

19. Tab 1: Facility Process List Facility Process List Lists all manufacturing and support processes associated with contract, program, or facility. This is not a list of surveillance requirements.

20. Tab 2: Risk Statement Generator Risk Statement Generator Identify risk consequence based on a predefined conditions and questions list to generate risk statements.

21. Tab 3: Performance Factors (All Suppliers) Performance Factors (All Suppliers) Identifies potential causes of the risk statements based on the supplier’s current and past performance. Applicable to all suppliers [Federal Acquisition Regulation (FAR) 52.246-2 and -11].

22. Tab 4: Performance Factors (Higher Level) Performance Factors (Higher Level) Identifies additional potential causes of the risk statements based on the supplier’s current and past performance. Applicable to FAR 52.246-11 suppliers.

23. Tab 5: Risk Profile & Plan Risk Profile and Plan Provides the minimum documentation of the applicability of specific risk factors and documents the relationship between risk factors and planned GCQA surveillance.

24. Tab 6: Sample Risk Profile Plan Sample RPP Examples and explanatory notes.

25. Tab 7: Update Log RP&P Update Log Record of RPP updates, revisions ……. RPP configuration management.

26. Tab 8: DC&A Plan DC&A Plan Results of data collection and analysis are used as a key component for the Performance Factor Assessment.

27. Tab 9: Risk Cause Likelihood Risk Cause Likelihood Used to assess the likelihood that the identified risk cause will occur.

28. Question and Answer Which tabs are used to identify the causes of risk statements based on the supplier’s current and past 12 months’ performance? (Select all that apply.) • Risk Statement Generator • Performance Factors (All Suppliers) • Performance Factors (Higher Level) • Risk Profile

29. Initiating a Risk Profile Lesson Topics: Importance of QA Risk Assessment The QA Risk Assessment Process Initiating a Risk Profile Developing a Facility Process List Risk Consequence Writing Risk Statements Identifying Potential Causes of Risks Assessing Risk Likelihood Developing a Data Collection & Analysis Plan Examining Risk Assessment Results

30. Topic 3: Initiating a Risk Profile Risk profile header information • Minimum requirements • Exceptions

31. Initiate Risk Profile Header • Minimum documentation required • Basic supplier information • Plan Type (program, contract, or facility) • Quality Management System (QMS) requirement

32. Exceptions A Risk Profile may not be required for subcontract Letters of Delegation (LODs) received to verify or witness specific tasks • Activities in the delegation serve as the surveillance plan LOD Select graphic for a sample LOD

33. Exceptions (cont.) Specific mandatory requirements in Quality Assurance Letters of Instruction (QALIs) or LODs do not require a risk assessment • DCMA QA responsibility for prime contracts are not limited to the QALI requirements • Specific requirements shall be identified in the surveillance plan QALI Select graphic for a sample QALI

34. Exercise 1:Risk Profile and Plan Header • Students work in pairs to complete the Risk Profile and Plan Header information in the Risk Profile & Plan tab. • One student open the Blank_Risk_Profile_and_Plan.xlsx file to record the information. • One student open: • Completed Contract Technical Review (CTR) results (pdf) file from Module 3, Lesson 1 • ValleyForgings_Contract.pdf • Use the Contractor and Government Entity Code (CAGE) entered in the Module 1, Lesson 1 CTR Exercise • Save Risk Profile and Plan with a new name for use in later exercises • Time: 10 minutes

35. Developing a Facility Process List Lesson Topics: Importance of QA Risk Assessment The QA Risk Assessment Process Initiating a Risk Profile Developing a Facility Process List Risk Consequence Writing Risk Statements Identifying Potential Causes of Risks Assessing Risk Likelihood Developing a Data Collection & Analysis Plan Examining Risk Assessment Results

36. Topic 4: Developing a Facility Process List • Facility Process List identifies processes suppliers use to: • Design • Produce • Deliver • Facility Process List is a tab on the Risk Profile and Plan Tool

37. Facility Process List • List is part of the Risk Profile and Plan Tool spreadsheet • QMS may be listed as a single process or individual clauses • Based on flow of product/data through the supplier’s facility • Begin at packaging/shipping • Work backward through all manufacturing and/or support processes • Develop a Flow Chart (Recommended)

38. Facility Process List - Support Processes Support processes are: • Supplier processes that support some or all of the manufacturing processes required for product realization • The output of support processes do not result in product characteristics • Examples of process include: • Control of purchases • Material Inspection Receiving Report (MIRR) preparation • Other similar support processes • Wide Area Workflow (WAWF) or DD-250

39. Facility Process List - Special Processes Special processes are any processes where the resulting output cannot be verified by subsequent measurement. • Examples of process include: • Shot peening • Nondestructive Testing (NDT) • Liquid penetrant • Magnetic particle • Ultrasonic • Radiography

40. Facility Process List - Stryker Armored Vehicle Example

41. Exercise 2:Develop Facility Process List • Pair with previous partner to develop the Facility Process List in the Facility Process List tab. • One student open the saved Risk Profile and Plan Tool (xlsx) file to record the information. • One student open: • Completed CTR results (pdf) file from Module 3, Lesson 1 • ValleyForgings_ManufacturingFlowChart.pdf • ValleyForgings_Traveler.pdf • Save the Risk Profile and Plan • Time: 15 minutes

42. Risk CONSEQUENCE Lesson Topics: Importance of QA Risk Assessment The QA Risk Assessment Process Initiating a Risk Profile Developing a Facility Process List Risk Consequence Writing Risk Statements Identifying Potential Causes of Risks Assessing Risk Likelihood Developing a Data Collection & Analysis Plan Examining Risk Assessment Results

43. Topic 5: Risk Consequence Assess risk consequence by using the Risk Statement Generator tab in the Risk Profile and Plan tool.

44. Assess Risk Consequence • Risk Statement Generator tool • Contains the indicators for high and moderate risk consequence • Low risk consequence is the result with no high or moderate conditions

45. Determine the applicability of each indicator to the contract, facility, and/or product/service

46. Assess Risk Consequence (cont.) • Contract Technical Review provides basis for identifying risks   • Contract or purchase order • Key or critical characteristics or processes • Memorandum of Agreement (MOA), QALI, or LOD • Drawings and specifications • Risk consequence assessment need not be repeatedfor each contract, unless additional requirements add an additional risk statements CTR eTool Manual

47. Exercise 3:Complete a Risk Consequence Assessment • Pair with previous partner to assess Risk Consequence in the Risk Statement Generator tab. • One student open the saved Risk Profile and Plan Tool (xlsx) file to record the information. • One student open: • Completed CTR results (pdf) file from Module 3, Lesson 1 • ValleyForgings_ManufacturingFlowChart.pdf • ValleyForgings_Traveler.pdf • Save the Risk Profile and Plan. • Time: 15 minutes

48. Writing Risk Statements Lesson Topics: Importance of QA Risk Assessment The QA Risk Assessment Process Initiating a Risk Profile Developing a Facility Process List Risk Consequence Writing Risk Statements Identifying Potential Causes of Risks Assessing Risk Likelihood Developing a Data Collection & Analysis Plan Examining Risk Assessment Results

49. Topic 6: Writing Risk Statements Risks statements describe what the QAS wants to avoid.

50. For each "yes" answer, write a Risk Statement or revise the Risk Statement to answer the question: NA NA High Automatic High Risk Impact Indicators “What do we want to make sure doesn’t happen?” 1. Is the product identified as a Aviation or Ship Critical Safety Item? Yes No • 2. Is the product a Non-Aviation or Non-Ship CSI? E.g. Life Support, Body Armor, Vehicle Armor, etc. Yes No 3. Is the contract subject to Agency Level Memorandum of Agreement (MOA), Delegation of Authority (DOA) that include commitments associated with a high risk characteristic or process, e.g. Space & Missile MCI, Ammunition? Yes No 4. Do the product specifications identify criticalcharacteristics or processes? Yes No Example: Risk Statement Generator (1 of 3) Stryker Armored Vehicle Example Supplier fails to control processes impacting ballistic capability. Supplier fails to control critical characteristics/processes.