Monitoring the Effectiveness of the LANL Operating Experience Program

1. Monitoring the Effectiveness of the LANL Operating Experience Program Robert B Stuewe Operating Experience Program Coordinator Los Alamos National Laboratory May 14, 2013

2. Scope of the LANL OPEX ProgramServes 11,000 people including the Los Alamos Field Office • OPEX Program Stats • 700-800 items processed per year • Multiple pre-planned sources • 50% from DOE (all of you) • All program/function areas • >100 internal institutional per year • 1500-3000 unique users/month • >5000 in searchable archive • >12,000 doc accesses/month • >200 searches/month (up to 400)

3. OPEX Monitoring and EvaluationUse Assessments, Measures, Analysis, Stakeholder Feedback • Annual • Assess OPEX Program Implementation and Effectiveness (part of Annual CAS Assurance Process) • Lessons Learned Improvement Workshop (includes Field Office) • Semi-Annually (or more frequently) • Lessons Learned Effectiveness Monitoring (often includes Field Office) • Quarterly • ORPS/Sub-ORPS Trending and Analysis • Monthly • LANL Dashboard • Daily/Weekly • Lessons Learned Archive Analytics • As Occur • Other Internal and External Assessments (3-5 per year) Lessons Learned Images: Adult Mexican Spotted Owl recorded on LANL property in June 2011. In compliance with the LANL Habitat Management Plan, work with heavy equipment must stop in areas of sensitive habit beginning March 1.

4. Annual CAS Management Self-Assessment • Part of LANS Board of Governors Approved Contractor Assurance System • DOE O210, O 226, and Prime Contract H-4 Clause set primary requirements • Uses feedback from interviews, measures, issues management system, LLEM, and other assessments • Results used by the Annual ISM Verification • OPEX Implementation • Processes are designed to reliably fulfill requirements, and are used by functional, programmatic , and organizational areas important to the site’s mission • OPEX Effectiveness • “There is objective evidence that experience from operational events is being tracked and used to drive continuous improvement” NNSA NAP-21 Element 2 Effectiveness Criteria Lessons Learned in Images:A subcontractor employee at LANL suffered the loss of a fingertip on 2/9/12, when he caught a finger in the bolt hole shown above during the repair of a bucket on a skid-steer loader.

5. The LANL OPEX Program is compliant, implemented, and is demonstrating effectiveness, but opportunities to improve effectiveness still exist The use of lessons learned for work planning could be improved Variability is evident in the lessons learned processes used by sub-units The rate of sharing lessons learned across the Laboratory is at an acceptable level but is trending unfavorably Monitoring and Evaluation ResultsSummarizing results from the last two years These opportunities for improvement are likely evident across the DOE.

6. Many Lessons Learned are not easy to use for work planning Too general or truisms (e.g., Always follow procedures and you won’t have problems) Too site specific (e.g., Procedure GG-XY-0010-1.1 needs to be changed) Do not include actions that others could use to prevent the adverse event Does not describe the activity or equipment that was associated with the lesson in a manner that enables quick relevance determination during work planning Redundancy and confusion in DOE LL directives and processes (When to use which PM lessons learned method, DOE-EM lessons learned process) DOE-Wide Contributing Factors Need to reduce cost-to-share while also creating higher-quality lessons for our activities?

7. Additional Details

8. Quarterly Lessons Learned Effectiveness Monitoring (LLEM) • Trending/analysis suggests that an accelerated rate of learning is needed • OPEX Program Coordinator selects the focus area, approves the data collection plan • OPEX Program personnel go into field to collect data (Field Office Counterpart can and does shadow) Lessons Learned in Images: On 8/9/10 at TA-54 Area L, a 4-liter glass container of nitric acid began to vent a red vapor from inside a 14-gallon poly drum. The drum was closed and the area evacuated. No workers were exposed. HAZMAT responded and handled the cleanup. GloveboxSafety Lessons Learned Effectiveness Use of LL in Training Development and Delivery R&D Safety Improvement Effectiveness Evaluation Project Management Lessons Learned Procedure Use of LL in Maintenance Work Planning Use of LL in Maintenance Pre-Job Briefs Crane Inspection Card Maintenance LL Follow-up

9. OPEX LANL Dashboard MeasuresUpdated Monthly, Action Plans for Red and Yellow levels, accessible by Field Office to support oversight • Processing Time • Urgent within 24 hours • Caution within 7 days • Knowledgebase Inventory • Internal LL Generation Rate • Document Access Rate • Search Rate Lessons Learned Images:Plumber’s kit containing propane bottle discovered 3/6/12 at the LANL CMR facility during the performance of an extent of condition immediate action following a TSR violation that had resulted from the discovery of another propane bottle on 2/15/12. Staying Green requires monitoring knowledgebase analytics on daily and weekly bases, and taking immediate action before monthly measures suffer!