Mitigating Organizational Weaknesses Presented September 13, 2006 DOE Integrated Safety Management Best Practices Wor

1. 1 Background EPC attitude (flip chart) Get �er done We�ve always done it that way Focus on schedule/cost variance Who is at fault Traditional tried & true processes such as RCA To be competitive, Bechtel implemented PBL in the late 90�s and Six Sigma in 2000 � integrated ABC & Y=f(x) WTP challenge � significant staffing challenge, 1800 in approximately 1 yr, different GBUs, new hires, etc... Few NQA-1 qualified vendors Frequent action to close a CAR is to revise procedure On WTP, quality, safety and productivity improvements were addressed through six sigma and PBL Six Sigma Black Belts were included as members or leads of all RCA�s since 2003. Background EPC attitude (flip chart) Get �er done We�ve always done it that way Focus on schedule/cost variance Who is at fault Traditional tried & true processes such as RCA To be competitive, Bechtel implemented PBL in the late 90�s and Six Sigma in 2000 � integrated ABC & Y=f(x) WTP challenge � significant staffing challenge, 1800 in approximately 1 yr, different GBUs, new hires, etc... Few NQA-1 qualified vendors Frequent action to close a CAR is to revise procedure On WTP, quality, safety and productivity improvements were addressed through six sigma and PBL Six Sigma Black Belts were included as members or leads of all RCA�s since 2003.

2. 2 WTP Environment 2001 - 2005 2001 challenges � hired/transferred 1800 in approximately 1 yr. from different Bechtel groups, new hires, staff augs Majority of the technical training was required reading Few NQA-1 qualified vendors Corrective action improvements consistently included procedure revisions Quality, safety and productivity improvements were addressed through six sigma and PBL Bechtel implemented Performance Based Leadership in the late 90�s Six Sigma in 2000 Both to improve productivity, quality of work Six Sigma Black Belts were included as members or leads of all RCA�s since 2003. Large reduction in force May 05 and November 05

3. 3 WTP Environment 2001 - 2005 Did latent organizational weaknesses exist?

4. 4 WTP Environment 2001 - 2005 2001 challenges � hired/transferred 1800 in approximately 1 yr. from different Bechtel groups, new hires, staff augs Majority of the technical training was required reading Few NQA-1 qualified vendors Corrective action improvements consistently included procedure revisions Quality, safety and productivity improvements were addressed through six sigma and PBL Bechtel implemented Performance Based Leadership in the late 90�s Six Sigma in 2000 Both to improve productivity, quality of work Six Sigma Black Belts were included as members or leads of all RCA�s since 2003. Large reduction in force May 05 and November 05

5. 5 Industry Event Causes Ask if front-line workers account for the bulk of causes of significant events. True or False? If you were looking at the raw data here what would you conclude? Ask if all other cause categories were added, what do we have? Organizational and management factors. Nonwork practices: In the chart, 68% of causes of all significant events are attributable to organization and leadership. Fact: Other sources indicate that the division is 80% organization and 20% individual (Rummler & Brache). Ask if front-line workers account for the bulk of causes of significant events. True or False? If you were looking at the raw data here what would you conclude? Ask if all other cause categories were added, what do we have? Organizational and management factors. Nonwork practices: In the chart, 68% of causes of all significant events are attributable to organization and leadership. Fact: Other sources indicate that the division is 80% organization and 20% individual (Rummler & Brache).

6. 6

7. 7 Fall 2005 Nuclear Safety and Quality CultureBNI, DOE ORP and Office of Enforcement agreed WTP had weaknesses in: Procedure compliance Adequacy of procedures Training Balance of cost, schedule, and quality Communication and feedback Quality improvement Questioning attitude Management behaviors Assessment and oversight Balancing safety, quality, cost and schedule Verification of work activities

8. 8 Achieving nuclear safety and quality through procedure compliance Strategy: Bottom up and top down Focused Working with employees to establish ownership Strategy: Bottom up and top down Focused Working with employees to establish ownership

9. WTP Nuclear Safety and Quality Imperative

10. 10 Nuclear Safety and Quality Imperative Project We are approaching this problem as though it was a project. We have defined the scope of work, assigned a senior manager and other organizational resources, developed an execution schedule, and have established metrics to provide feedback and enable assessment of our progress. The goal of this quality project is to ensure ongoing work is of adequate quality while we establish robust long term processes and most importantly create a work force with an enduring Nuclear Quality and Safety Culture. � On the wall you see several important charts that we will reference during this discussion. I would like to call your attention to the Nuclear Safety and Quality Work Breakdown Structure. This document is the foundation of our Quality Improvement Project. � � � We are approaching this problem as though it was a project. We have defined the scope of work, assigned a senior manager and other organizational resources, developed an execution schedule, and have established metrics to provide feedback and enable assessment of our progress. The goal of this quality project is to ensure ongoing work is of adequate quality while we establish robust long term processes and most importantly create a work force with an enduring Nuclear Quality and Safety Culture. � On the wall you see several important charts that we will reference during this discussion. I would like to call your attention to the Nuclear Safety and Quality Work Breakdown Structure. This document is the foundation of our Quality Improvement Project. � � �

11. 11 Culture change strategy Moving our organizational culture to a level where our work is consistently performed to the highest level of nuclear quality is our top priority. We are approaching this change with a number of top-down actions as well as through many specific tactical actions taken at each functional level. Initially, our effort looked and felt a little disjointed. It was confusing to us and to our workforce and the results were disappointing. Over the past several months though, our efforts have coalesced into a structured approach that I believe will re-establish the confidence in our program and our people and result in overall project success. Moving our organizational culture to a level where our work is consistently performed to the highest level of nuclear quality is our top priority. We are approaching this change with a number of top-down actions as well as through many specific tactical actions taken at each functional level. Initially, our effort looked and felt a little disjointed. It was confusing to us and to our workforce and the results were disappointing. Over the past several months though, our efforts have coalesced into a structured approach that I believe will re-establish the confidence in our program and our people and result in overall project success.

12. 12 Project-wide compensatory action

13. 13 Centralized issue tracking system: PIER System implemented to ease identification of issues

14. 14 NSQ culture change activities are pervasive

15. 15 Desired management behaviors Balance safety and quality and cost and schedule Maintain consistent conservatism in decisions Be receptive to constructive criticism, willing to acknowledge mistakes, and open and transparent Willingness to actively listen, be influenced, provide feedback, interact with employees Give measured, reflective, facts-based responses Provide consistent, meaningful, positive reinforcement Apply fair and consistent discipline

16. 16 Human Performance Senior Management sponsorship � Larry Simmons Integrating HP into our current processes through employee education Self assessments CAR causal identification Event critiques Root Cause Analysis Employee Concerns and Employee Relations investigations Trained 69 people to date; 49 trained in the 4 day HPI course; 600+ trained at July Safety Rally (manuals & non-manuals); 2 trained at INPO Moving toward self-sufficient training Using EPC examples

17. 17 Procedures PIP � Process Phases: Identify Requirements Process Mapping Measure Fishbone XY Matrix Data Collection Analyze Analyzed Data Collected Review other sites procedures

18. 18 Procedures PIP � Process X�s (largest contributors): Human Factors Length Complexity Should vs. Shall Structure Inconsistencies

19. 19 Procedure improvements Identified potential improvement opportunities Compared procedures with Savannah River Site Improvement action items: Revision of Document Administration Procedure Standard structure Clearer definitions Include attributes of an adequate procedure Better flowdown and change management Evaluation/revision of AB/QAM affecting procedures Three-phased approach; Phase I nearing completion Attributes of an adequate procedure Identifies scope Identifies and defines interfaces (responsibilities, information, and deliverables) Defines division of responsibility Implements valid requirements Includes reasons and source for changes in revision history Attributes of an adequate procedure Identifies scope Identifies and defines interfaces (responsibilities, information, and deliverables) Defines division of responsibility Implements valid requirements Includes reasons and source for changes in revision history

20. 20 Training PIP Phases: Identify & Measure Requirements Existing Process Mapping Analyze Identify Process (XY Matrix) Design Detailed Process Mapping Verify Plans and Schedules

21. 21 Training PIP � Process Potential Solutions: Continue existing training program Systematic Approach to Training on Positions that meet criteria (Graded Approach) Systematic Approach to Training on all Positions Supervisor verifies competency

22. 22 Training improvements Prior Approach Manage/Supervisor accountability On-target; procedure-focused Narrow; limited classroom Basis not well documented No measure of effectiveness Interim Actions Discipline-specific enhancements to address identified shortcomings Overall objective Documented basis Broad perspective Effective media

23. 23 Engineering Actions Relevant management assessments: Safety Envelope Conformance Self-assessment Report For CIS Self-assessment 2006 Safety Envelope Maintenance Program Corrective Action Assessment� Safety envelope stand down conducted in July 2005 Personal Commitment to Safety Envelope Compliance in June 05. Safety envelope training is the CBT module. Knowledge check causes student to access the Design Criteria Database (DCD) and perform searches that identify the criteria relevant to answering questions about specific proposed design changes. �Technical criteria checklists� are in use by Mech Systems. Focus is on P&IDs. �Improved feedback� comment for design criteria changes refers to the impact assessment/voting button process used when changes in the DCD are implemented. Process institutionalized in Design Criteria EDPI Performance improvement noted following addition of ENS sign off on documents that require a safety screen. Relevant management assessments: Safety Envelope Conformance Self-assessment Report For CIS Self-assessment 2006 Safety Envelope Maintenance Program Corrective Action Assessment� Safety envelope stand down conducted in July 2005 Personal Commitment to Safety Envelope Compliance in June 05. Safety envelope training is the CBT module. Knowledge check causes student to access the Design Criteria Database (DCD) and perform searches that identify the criteria relevant to answering questions about specific proposed design changes. �Technical criteria checklists� are in use by Mech Systems. Focus is on P&IDs. �Improved feedback� comment for design criteria changes refers to the impact assessment/voting button process used when changes in the DCD are implemented. Process institutionalized in Design Criteria EDPI Performance improvement noted following addition of ENS sign off on documents that require a safety screen.

24. 24 Procedure compliance checklists Purpose: Leading indicator to identify error precursor conditions Implemented by document checker as part of normal, pre-approval, document review Used for calculations, drawings, specifications, material requisitions, and supplier deviation disposition requisitions Initial results: High originator compliance by attribute (0.25% checker �holds� for correction by originator) About 8% of design documents had at least one compliance �hold� for correction that was identified during checking Metrics pinpoint actionable areas of improvement opportunity Procedure compliance checklists are a Compensatory Action with respect to full development of NSQ culture changes. �Leading Indicators� because they are deployed in checking process (prior to document approval) Approx 1,100 checklists completed over 8 week period beginning 24 April 06 Total of 41,225 unique procedure compliance requirements are associated with the 1100 documents for which checklists have been completed to date (about 40 requirements per document). Directly relevant to weaknesses: Promotes/improves procedure compliance Improves knowledge of procedure requirements (training) Useful as an oversight tool Allows checkers greater focus on technical attributes and supports a �zero defect� criteria when documents are submitted for approval Previous efforts to improve calculation quality have proven beneficial, as originators have been conditioned to utilize the checklists in preparing the document. �Checker holds� for calculations is lowest of all document types (< 6%) Documents that represent interrelated requirements or that combine processes from one or more EDPIs, although produced in smaller numbers, have a higher non-compliance score. Document originators are also using the checklists during document preparation. This is a good practice and is being encouraged. Issues identified by checkers generally did not accumulate in any specific procedural requirement, indicating that procedure clarity may not be a problem, however, more data is necessary to make that determination. �Actionable areas� currently focus on one-on-one or group coaching to improve understanding of requirements. Checklists are capable of pointing out the need for discipline-specific instruction to improve performance by originators. (See next page)Procedure compliance checklists are a Compensatory Action with respect to full development of NSQ culture changes. �Leading Indicators� because they are deployed in checking process (prior to document approval) Approx 1,100 checklists completed over 8 week period beginning 24 April 06 Total of 41,225 unique procedure compliance requirements are associated with the 1100 documents for which checklists have been completed to date (about 40 requirements per document). Directly relevant to weaknesses: Promotes/improves procedure compliance Improves knowledge of procedure requirements (training) Useful as an oversight tool Allows checkers greater focus on technical attributes and supports a �zero defect� criteria when documents are submitted for approval Previous efforts to improve calculation quality have proven beneficial, as originators have been conditioned to utilize the checklists in preparing the document. �Checker holds� for calculations is lowest of all document types (< 6%) Documents that represent interrelated requirements or that combine processes from one or more EDPIs, although produced in smaller numbers, have a higher non-compliance score. Document originators are also using the checklists during document preparation. This is a good practice and is being encouraged. Issues identified by checkers generally did not accumulate in any specific procedural requirement, indicating that procedure clarity may not be a problem, however, more data is necessary to make that determination. �Actionable areas� currently focus on one-on-one or group coaching to improve understanding of requirements. Checklists are capable of pointing out the need for discipline-specific instruction to improve performance by originators. (See next page)

25. 25 Procedure compliance checklists (examples) This slide illustrates the utility of the checklists: Across all discipline drawing checklists, questions 13, 24 and 32 were missed most frequently (with a noticeable spike on 32). Dev 13 ) �Drawing/DCN is consistent with other design documentation (i.e., P&IDs, V&IDs, data sheets, specifications).� Dev 24 ) �Drawing incorporates identification and location of equipment, and current supplier designs as applicable.� Dev 32 ) �Revisions are clearly identified such as by placing a cloud around the change and marking with revision triangles.� (Note that this was an element of Quality Level (spool) issue and highlighted in OE letter.) For Electrical detail chart: Dev 32 ) �Revisions are clearly identified such as by placing a cloud around the change and marking with revision triangles.� Dev 13 ) �Drawing/DCN is consistent with other design documentation (i.e., P&IDs, V&IDs, data sheets, specifications).� Dev 24 ) �Drawing incorporates identification and location of equipment, and current supplier designs as applicable.� Dev 16 ) �Drawing numbering is in accordance with 24590-WTP-GPP-PADC-001, WTP Document Numbering.� Dev 20 ) �Drawings were checked for resolution of identified issues identified in the 3-D Model, and any revisions were appropriately completed.� This slide illustrates the utility of the checklists: Across all discipline drawing checklists, questions 13, 24 and 32 were missed most frequently (with a noticeable spike on 32). Dev 13 ) �Drawing/DCN is consistent with other design documentation (i.e., P&IDs, V&IDs, data sheets, specifications).� Dev 24 ) �Drawing incorporates identification and location of equipment, and current supplier designs as applicable.� Dev 32 ) �Revisions are clearly identified such as by placing a cloud around the change and marking with revision triangles.� (Note that this was an element of Quality Level (spool) issue and highlighted in OE letter.) For Electrical detail chart: Dev 32 ) �Revisions are clearly identified such as by placing a cloud around the change and marking with revision triangles.� Dev 13 ) �Drawing/DCN is consistent with other design documentation (i.e., P&IDs, V&IDs, data sheets, specifications).� Dev 24 ) �Drawing incorporates identification and location of equipment, and current supplier designs as applicable.� Dev 16 ) �Drawing numbering is in accordance with 24590-WTP-GPP-PADC-001, WTP Document Numbering.� Dev 20 ) �Drawings were checked for resolution of identified issues identified in the 3-D Model, and any revisions were appropriately completed.�

26. 26 Procedure compliance checklists Actions based on metrics: Continue using checklists for all calculations, drawings, specifications, material requisitions, and supplier deviation disposition requests Discipline-specific actions based on metrics: Originator feedback for non-recurring checker �holds� Staff coaching for recurring checker �holds� Procedure clarification Detailed discipline-specific metrics have been prepared for the disciplines. This action is in progress by the DSLs. Based on indication that inter-related processes, and processes not used as often (i.e., MRs SDDRs) both benefit from compliance checklists. Evaluate expansion of procedure compliance checklists to other document types. A number of Commercial Grade Dedication (CGD) CARs were issued recently; it may be advisable to generate a checklist for application of CGD. Detailed discipline-specific metrics have been prepared for the disciplines. This action is in progress by the DSLs. Based on indication that inter-related processes, and processes not used as often (i.e., MRs SDDRs) both benefit from compliance checklists. Evaluate expansion of procedure compliance checklists to other document types. A number of Commercial Grade Dedication (CGD) CARs were issued recently; it may be advisable to generate a checklist for application of CGD.

27. 27 Potential At Risk Practices Performing a task with two or more procedures Following procedures cookbook style Removing several danger tags quickly without annotating removal of the tags on the clearance sheet Performing critical checks without peer review or concurrent verification Having several review and approval steps Attempting to lift too much weight to avoid multiple trips Signing off several steps of a work plan or procedure before performing the steps

28. 28 Construction Actions

29. 29 Changing our behavior Safety leadership development training Communications Planning Leading indicators and metrics Motivation Revising site work rules and discipline policy with the Safety Alliance Implementing HPI Application process for VPP Star Status Specific goals in key managers� annual reviews and performance measurement plans

30. 30 Events Occur . . . More often due to error-prone tasks and error-prone work environments than from error-prone individuals. Error prone tasks and work environments are usually created by latent organizational weaknesses.

31. 31 How do we measure culture? We have contemplated numerous metrics to help us assess how we are doing. You have seen some results from our employee surveys, and you can see we have two primary metrics on the wall that measure significance of our events and the point that they were identified. We have considered the school of thought that says an increase in the level of self identified problems is a positive indicator because it demonstrates an active assessment program and a questioning attitude. Conversely it may also indicate deteriorating quality performance. The reverse argument can be made for a metric that shows a reduction in identified quality problems. The metrics we have settled on enable us to analyze the significance of the issue and when and how the issues are being identified. Our goal is to see the significance of the problems diminish and the identification of the problem occur earlier in the process. � � We have contemplated numerous metrics to help us assess how we are doing. You have seen some results from our employee surveys, and you can see we have two primary metrics on the wall that measure significance of our events and the point that they were identified. We have considered the school of thought that says an increase in the level of self identified problems is a positive indicator because it demonstrates an active assessment program and a questioning attitude. Conversely it may also indicate deteriorating quality performance. The reverse argument can be made for a metric that shows a reduction in identified quality problems. The metrics we have settled on enable us to analyze the significance of the issue and when and how the issues are being identified. Our goal is to see the significance of the problems diminish and the identification of the problem occur earlier in the process. � �