ORR/RA Criteria and Review Approach Document (CRAD)

1. ORR/RA Criteria and Review Approach Document (CRAD) Developing an ORR/RA CRAD An Interactive Exercise May 2005 Revision 5T5L

2. Upon the completion of this Interactive Exercise module, you should be able to: Using the Core Requirements of DOE O 425.1C and the approved Plan of Action (POA), develop review Objectives. Select appropriate requirements as Criteria that pertain to the Objectives from rules, contractual orders and standards, and other A/B List requirements. Develop activities for the Implementation Plan (IP) Approach to evaluate the Criteria in a performance based manner. 5T5L May 2005 Revision

3. Principles . . . Flowdown • 1. Requirements > • DOE/NNSA Order Core Requirements and Plans of Action. • 2. Develop Objectives > • Based on Team size and expertise. Drawn from the Requirements (#1) in executable pieces. Usually grouped in Functional Areas as Team Members generally come with “functional area” expertise. 5T5L May 2005 Revision

4. Principles . . . Flowdown (continued) 3. Establish Criteria > -Based on Requirements levied on the Contractor via law or contractual agreement by the DOE/NNSA. - May be modified by Plan of Action (POA). 4. Select Review Approaches > - Documents that implement requirements (how they say it is done), - People that impact the process (who gets it done), and - Operations (where it all comes together). 5T5L May 2005 Revision

5. CRADs • Criteria and Review Approach Document (CRAD) delineates: Objective, Criteria with References, and Approach for the review. • Through the ORR / RA review: • Records Review, Interviews, and Observations are completed in accordance with “Approach.” • If this review indicates that these elements are adequate/satisfactory, then “Criteria” are met. • If all Criteria are met, the “Objective” is met. • Identified “Findings” indicate deficiencies where all of the Criteria and/or Objective is not fully met. 5T5L May 2005 Revision

6. Example of a Bad CRAD: Objective and Criteria • TRAINING (TR) • Objective: The Site has an excellent training program. • Criteria: • -Training covers all current and all important topics. • - All personnel have met all training requirements. • - Training complies with all National Education Association (NEA) standards. 5T5L May 2005 Revision

7. Example of a Bad CRAD Approach Approach: Record Review: - Read all training Records provided by the Site and verify that they meet all NEA standards. - Review training records for all personnel and validate that they are all current. Interviews: - Interview persons responsible for establishing training requirements. - Interview instructors. 5T5L May 2005 Revision

8. Example of a Bad CRAD Approach continued Approach: Observations: - Observe all Site training provided during the review period. - Watch any On-the-Job Training (OJT). 5T5L May 2005 Revision

9. Interactive CRAD Exercise - Objectives 1. Form “groups” of 3 to 5 people. 2. Review the excerpts of the Provided Order and the Plan of Action (POA). 3. Develop Objectives for Training (TR) area Implementation Plan (IP). 4. We will discuss this part of the exercise in 15 minutes. 5T5L May 2005 Revision

10. CRAD Exercise: Objectives • The TRAINING Objectives for this Facility and trained crew, for a new activity at Oak Ridge (OR) Y-12 Building 9212 Wet Chemistry Operations ORR (Rev 0 of August 2002) come from the Plan of Action (POA) excerpts of Sections 2.0 (Facility Description, page 1), 6.0 (Restart Discussion, page 2), and 7.0 (Proposed Breadth, pages 2-9) • See OR ORR IP (Rev 0 of 7 October 2002) excerpts: Training Objectives TR.1 and TR.2 (pages 37-39) . . . “Training and Qualification Programs . . . and Modifications to the facility for training and qualification. (Core Requirements 3 and 5 (425.1C))”. 5T5L May 2005 Revision

11. Interactive CRAD Exercise - Criteria 1. Return to your “group.” 2. For the Objectives provided on the previous slide, review the excerpts of the training directives provided. 3. Establish (and Reference!) Criteria for the 2 Training Objectives. 4. We will discuss this exercise in 15 minutes. 5T5L May 2005 Revision

12. CRAD Exercise: TR.1 and .2 Criteria • See OR ORR IP (Rev 0 of 7 October 2002) excerpts: Training Objectives TR.1 and TR.2 (pages 37-39) . . . “Training and Qualification Programs . . . and Modifications to the facility for training and qualification. (Core Requirements 3 and 5 (425.1C))”. 5T5L May 2005 Revision

13. Interactive CRAD Exercise - Approach 1. Return to your “group.” 2. For the Criteria provided on the previous slides develop an Approach to evaluate documentation, interview personnel, and observe operations and equipment that will allow a reasonable judgment of the Criteria for the 2 Training Objectives. 3. We will discuss this exercise in 15 minutes. 5T5L May 2005 Revision

14. CRAD Exercise: TR.1 and .2 Approach • See OR ORR IP (Rev 0 of 7 October 2002) excerpts: Training Objectives TR.1 and TR.2 (pages 37-39) . . . “Training and Qualification Programs . . . and Modifications to the facility for training and qualification. (Core Requirements 3 and 5 (425.1C))”. 5T5L May 2005 Revision

15. Examples: Good CRAD, “OR Y-12 Wet Chemistry ORR” Training - 10/02 • “OR Y-12 Wet Chemistry ORR” Training CRAD Objectives, Criteria, and Approaches (DOE O 425.1C). • Key Principles for a CRAD: • Objectives come from Core Requirements and Plan of Action; • Criteria based on Requirements: Law, Contract, A/B List, etc… • Approach to address Criteria and able to be completed in allotted time. 5T5L May 2005 Revision

16. CRAD Development and Use Summary • Objectives and Criteria meet the ORR / RA Scope and Breadth (POA). Criteria are requirements-based (and referenced). • Approach delineates ORR / RA Depth. • Approach: Record Review, Interviews, Observations met? >Criteria met? >Objectives met? = Successful ORR/RA. 5T5L May 2005 Revision