Nuclear Industry Evaluation Program

1. Nuclear Industry Evaluation Program Insights To Influence Performance Excellence At Operating Nuclear Power Plants David A. Taggart Senior Member, ASQ

2. Diablo Canyon Power Plant Diablo Canyon Power Plant, located in San Luis Obispo County, provides electricity for more than 2 million northern and central Californians from its two 1,100 megawatt units.

3. Humboldt Bay Power PlantLocated Near Eureka California

4. Insights to Influence Performance Excellence - Quality Assurance • QA identify weaknesses that may be blind to the organization. • QA provides insights and feedback on the values and principals by which organizations operate. • QA organizations place “eyes on the problem” and provides an opportunity to reinforce standards.

5. Insights to Influence Performance Excellence - Quality Assurance • Management needs to leverage their quality organizations to help in establishing a culture of operational excellence.

6. Insights to Influence Performance Excellence - Quality Assurance • Quality organizations need to reflect a diagonal slice of the overall nuclear organization as a whole. • Quality organizations require a good technical foundation with direct line management experience in conjunction with a broad based knowledge of the quality field and an understanding how to apply this knowledge to each functional area of the organization. • Requires the plant line-organization staff to be self critical.

7. Insights to Influence Performance Excellence - Quality Assurance • Quality organizations and line-management need to also leverage external peer resources to ensure stations standards are current to the industry. • Quality organizations need to exhibit a questioning attitude by challenging assumptions, investigating anomalies and considering potential adverse consequences. • Quality organization contribute through diversity of thought and serve as prevention to organization “groupthink.”

8. Insights to Influence Performance Excellence • Willingness to be self-critical and to make the necessary changes is essential for long-term success. • Have the courage to undertake new approaches. • Avoid complacency and cultivate a continuous learning environment. • Company Executives and Plant Managers must be willing to get and receive perceptions as well as facts.

9. Insights to Influence Performance Excellence • Quality needs to uphold the standards for excellence. • Improvement comes from an introspective search for performance shortfalls.

10. Insights to Influence Performance Excellence • Uncover the problem areas and relentlessly follow through until the problems are addressed. • A commitment to comprehensive independent audits in conjunction with line-management self-assessments is fundamental for achieving and maintaining operational excellence. Continue to reflect on the mix of the two.

11. NQML & NIEP Background • Nuclear Utilities with Operating Power Plants Established Peer Group of QA Directors • Nuclear Operating Plants Require a Periodic Evaluation of the QA Organizations on a Two Year Frequency • Uniform Excellence Criteria Established By Nuclear Industry Quality Organizations

12. Nuclear Quality Management Leadership Forum • The Nuclear Quality Management Leadership (NQML) forum consists of utility quality management decision-makers to sponsor and promote activities which support effective quality management within the nuclear industry. • The NQML optimizes the interface with other industry organizations to bring consistent and complimentary focus to the current issues challenging our business. • The NQML has partnered with the Nuclear Energy Institute to further promote the advancement of effective independent oversight of Nuclear Power operations.

13. Establishment of NIEP • The NQML has established an industry Nuclear Industry Evaluation Programs subcommittee to provide the focus and oversight of the independent assessment process. • The subcommittee is responsible for the development of good practices pertaining to successful oversight and implementation of the quality management function. • The subcommittee is responsible for the development of specific performance objectives and attributes associated with a leading nuclear industry quality management function and the development of industry peer evaluation guidelines.

14. NQML 07-001 “Nuclear Industry Evaluation Program Performance Objectives and Criteria” • Describes specific criteria to assist nuclear utilities and facilities in developing programs and improving their quality management and nuclear oversight functions. • The performance objectives are broad in scope and each objective lists several criteria to provide the breadth and depth of the objective.

15. NQML 07-002“Nuclear industry Evaluation Program” Establishes a uniform industry to evaluate the effectiveness of each member’s independent nuclear oversight function.

16. Governing Documents • NQML 07-002, Revision 1, Dated January, 2007, Effective 3/1/07, “Nuclear industry Evaluation Program” • NQML 07-001, Revision 1, Dated January, 2007, Effective 3/1/07, “Nuclear Industry Evaluation Program Performance Objectives and Criteria”

17. NIEP Performance Objectives • Quality Organization & Organization Effectiveness (A) • Internal Audit and Assessment Program (B) • Oversight of Performance Improvement Programs (C) • Independent Review Process (Off-site & On-site) (D) • Safety Culture (E) • Supplier Quality (F) • Independent Inspection (G) • Quality Assurance Program (H)

18. Example Objective & Attribute Objective A – Quality Organization & Organizational Effectiveness The Quality organization responsibilities and accountabilities for oversight are clearly defined, understood, and effectively implemented. Quality Managers, by leadership, commitment, and example, establish high standards of performance and align the organization to provide effective oversight. Attributes: A.1 Station management is provided independent oversight that identifies precursors to problems, analyzes causes, and recommends solutions that support safe, reliable, and efficient operating plants. A.2 The quality organization is staffed appropriately and key Quality organization positions are filled based on individual development and demonstrated readiness to fulfill the responsibilities of the positions. Individuals assigned to perform independent monitoring and assessments have the necessary experience, training, skills, credibility, and authority to conduct the reviews, audits, and analyses. [OR.5 – 2] A.3 An effective training program is established for Quality Personnel. This will include indoctrination, initial, and proficiency training. Quality Managers and Supervisors receive continuing education, and reinforcement to maintain and improve technical, leadership and managerial skills. A.4 Quality objectives, expectations, responsibilities and authorities are aligned with the stations mission and are established and understood by the Quality Organization, line management and station personnel. A.5 The Quality organization is a learning organization as exhibited by self-assessments, benchmarking and improving to industry best practices.

19. Example Objective & Attribute Objective E – Safety Culture The Quality organization ensures that the plant staff in all aspects of plant operations considers nuclear plant safety as the overriding priority. Decisions and actions are based on this priority and Oversight follows-up to verify that nuclear safety concerns receive appropriate attention. Attributes: The Quality organization will monitor through audits, assessments, and oversight activities, the following: E.1 Station management supports strong Corrective Action, Self-Assessment and Operating Experience Program implementation. E.2 Line Management is responsive to Oversight issues and defensiveness to these issues is not tolerated. E.3 Reviews are conducted for those programs and processes that are important to safety. Modifications to systems important to safety are reviewed prior to implementation and any identified issues are appropriately dispositioned. E.4 Individuals and Managers at all levels in the organization demonstrate a strong safety culture through appropriate behaviors. E.5 The station implements a formal and effective method for operational decision making. E.6 The station periodically assesses the health of their safety culture.

20. NIEP Team Composition & Qualifications • Team Manager • Director, Manager, Supervisor with QA/Oversight background • Team Members • (Expectation) Mgr., Supv., Sr. level personnel in a quality organization • Consideration of a member with ATL certification • Host Peer • Senior Management Representative • (Should) Mgr., Director, VP in Quality/Oversight

21. Basic Duties/Responsibilities-Team Manager • Team Composition/qualifications • Pre-Evaluation visit (site/telecon) • Assigning scope items to Team Members • Developing schedule • Team Orientation • Daily meetings with Team • Developing/Approving Final Report

22. Basic Duties/Responsibilities-Team Members • Pre-Evaluation reviews of documents provided • Developing vulnerabilities/Focus areas • Performing evaluation in assigned areas • Working with Team to reach final conclusions • Providing checklists, report input and related documentation

23. Basic Duties/Responsibilities-Host Peer • Team Logistics • Providing Team needed documents • Ensuring Request Letter and Authorization completed and issued • Facilitating development and issuance of Plan • Participating in Team Meetings and Challenging issues to ensure accuracy

24. Basic Duties/Responsibilities-Senior Management Representative • Interfacing with Senior Site and Corporate Management • Participating in Evaluation of Assigned Objective(s), if directed by Team Manager • Providing Input to Interview Questions • Providing insights to evaluated member management

25. Basic Duties/Responsibilities-Evaluated Member • Performing a Pre-NIEP Self Assessment • Developing and approving Authorization Letter and Scope • Ensuring Oversight and Line support to Team • Approving NIEP Plan • Entering Issues into the company’s CAP • Providing Initial Team Briefing • Completing NIEP Feedback Questionnaire • Approving Team Composition

26. NIEP Evaluation Schedule per NQML 07-002 • Site Self-Assessment (T-?) • Team Manager Contact Evaluated Member for coordination (T-90 days) • Evaluated Member provide results of Self-Assessment (T-90 days) • Finalize Scope (T-60 days) • Finalize Team Composition (T-45 days) • Plan Finalized and Approved (T-30 days) • Host provide necessary documentation to Team (CD) (T-30 days)

27. Team Mgr Contact Host Site to set dates (T-6m) Team Mgr Contact Member Utilities & Solidify Team (T-6m) Team Mgr Contact Host Site & agree on scope (T-2m) Team Mgr distribution of data CD and assigns areas (T-1m) Host Site Conduct Self Assessment (T-12 m) Host Site Issue Authorization Letter & Scope (T-2m) Host Site Issue Plan including Attributes (T-2m) Host Site Provide data CD to Team Mgr (T-2m) NIEP PLANNING CHECKLIST

28. The evaluation team will meet the Sunday prior to the scheduled evaluation to: discuss the evaluation and to finalize their preparation prior to the scheduled entrance meeting Complete NIEP Team orientation to the team members by the Team Manager The evaluated member may participate to: provide the team with logistics of initial arrival at the station. Clarify any access and badging requirements NIEP Performance & Reporting

29. The evaluation team shall meet with the evaluated station in an entrance meeting to: confirm the evaluation scope present the evaluation plan introduce evaluation team members meet counterparts discuss the evaluation sequence and plans for the exit meeting confirm channels of communications (as required by the evaluated member’s quality program) NIEP Performance & Reporting

30. Each scheduled evaluation will normally require five (5) days onsite for each assigned evaluation team member. The evaluation shall include: Team observation of activities and interactions review of documentation and objective evidence interviews with affected personnel analysis of results In addition, management interviews with senior management and line organization’s managers/directors will be conducted as part of the evaluation process. NIEP Performance & Reporting

31. The Team will hold daily evaluation briefs within the Team to discuss observations and potential findings, recommendations, strengths, observations, perceptions and conclusions. The Team will bin identified issues by categories. Additional needed information will be identified for follow-up. The Host Peer should be in attendance. Daily debriefs shall be held with the evaluated organization to review the results of potential findings, recommendations, strengths, observations and conclusions. Identified actions and documents requests shall be identified. A pre-exit debrief may be conducted at the discretion of the Team Manager and the evaluated member management. NIEP Performance & Reporting

32. An exit meeting shall be held with the management of the evaluated utility, including both the line and the Quality organizations, to present the evaluation results. Exit meeting information results shall include an overall statement of effectiveness, a summary of each evaluated performance objective including its effectiveness, associated strengths, deficiencies and recommendations. Other insights may be communicated. NIEP Performance & Reporting

33. As part of the exit meeting (or soon after the meeting) the Team Manager should provide the evaluated member with a copy of the NIEP Feedback Questionnaire The Team Manager shall request that the evaluated member complete the form and return it to the NQML/NIEP Committee Chair within forty-five (45) days. NIEP Performance & Reporting

34. Within fifteen (15) days of the exit, the Team Manager shall provide a draft report to the Evaluation Team Members and to the evaluated member for review and feedback. Within thirty (30) days of the exit, the Team Manager will prepare a cover letter and approve the final NIEP Report documenting the evaluation results. An electronic copy of the Report shall also be provided to the NQML/NIEP Subcommittee Chair. NIEP Performance & Reporting

35. Within forty-five (45) days of the exit, the evaluated member responsible management shall provide the NQML/NIEP Subcommittee Chair with a copy of the NIEP Feedback Questionnaire. Typically the evaluation results are documented in the evaluated members report format. NIEP Performance & Reporting

36. As a minimum, the following will be included: Description of the evaluation scope Identification of the evaluators Persons contacted during the evaluation activities and the entrance and exit meetings A summary of the evaluation results, including a statement regarding the effectiveness of the objectives and attributes that were evaluated NIEP Performance & Reporting

37. Description of each identified issue to assure that the evaluated member can effectively carry out corrective action. Significant Deficiencies should be presented in a “Problem Development Summary” format. Problem Development Summaries should include: Significant Deficiency Problem Statement Examples/Supporting Details Potential Consequences Potential Causes/Contributors Additional Insights NIEP Performance & Reporting

38. It is the responsibility of the evaluated member to prioritize each issue and include them in their corrective action process when appropriate. Recommendations for correcting identified issues or improving the program as appropriate. NIEP Performance & Reporting

39. The following definitions should be utilized to assist with the classification of evaluation results into appropriate categories: Strength Significant Deficiency Deficiency Recommendation Oversight Performance Summary Effectiveness Summary NIEP Definitions