Description
An operational definition, also called functional definition,[1][2] is a result of the process of operationalization and is used to define something (e.g. a variable, term, or object) in terms of a process (or set of validation tests) needed to determine its existence, duration, and quantity.
Independent Oversight Assessment of the Portsmouth/Paducah Project Office Conduct of Operations Oversight of the Depleted Uranium Hexafluoride Conversion Plants
May 2012 Office of Safety and Emergency Management Evaluations Office of Enforcement and Oversight Office of Health, Safety and Security U.S. Department of Energy
Table of Contents
1.0 Purpose................................................................................................................................................... 1 2.0 Background ............................................................................................................................................ 1 3.0 Scope ...................................................................................................................................................... 2 4.0 Results ................................................................................................................................................... 2 5.0 Conclusions ........................................................................................................................................... 7 6.0 Findings and Opportunities for Improvement........................................................................................ 8 7.0 Follow-up Items ..................................................................................................................................... 9 Appendix A: Supplemental Information .................................................................................................. A-1 Appendix B: Documents Reviewed, Interviews, and Observations ........................................................ B-1
i
Acronyms
BWCS CA CAP CAQ CR CONOPS DOE DUF6 FIR FPD FR FRA FY HSS INPO IPT ISMS NQA NSOL PPPO QA SSO
Babcock & Wilcox Conversion Service LLC Corrective Action Corrective Action Program Condition Adverse to Quality Condition Report Conduct of Operations U.S. Department of Energy Depleted Uranium Hexafluoride Field Inspection Report Federal Project Director Facility Representative Functions, Responsibilities and Authorities Fiscal Year Office of Health, Safety and Security Institute of Nuclear Power Operations Integrated Project Team Integrated Safety Management System Nuclear Quality Assurance Nuclear Safety Oversight Lead Portsmouth/Paducah Project Office Quality Assurance Safety System Oversight
ii
Independent Oversight Assessment of the Portsmouth/Paducah Project Office Conduct of Operations Oversight of the Depleted Uranium Hexafluoride Conversion Plants
1.0 PURPOSE This report documents the U.S. Department of Energy (DOE) Office of Enforcement and Oversight (Independent Oversight), within the Office of Health, Safety and Security’s (HSS) independent assessment of the Portsmouth/Paducah Project Office (PPPO) oversight of conduct of operations (CONOPS) at the depleted uranium hexafluoride (DUF6) conversion plants. The Independent Oversight assessment focused on PPPO’s oversight of the identification, evaluation, analysis, transmittal, and closure of issues related to CONOPS at the DUF6 plants at the Portsmouth Gaseous Diffusion Plant, Piketon, Ohio, and the Paducah Gaseous Diffusion Plant, Paducah, Kentucky. The onsite portion of the review was performed March 12-16, 2012, by the HSS Office of Safety and Emergency Management Evaluations. The objectives of the Independent Oversight assessment were to evaluate: (1) the PPPO oversight program’s effectiveness in improving the contractor’s performance; and (2) the project office’s issues management processes.
2.0 BACKGROUND In 2010, DOE authorized hot functional testing of the two DUF6 conversion plants. On September 30, 2011, Babcock & Wilcox Conversion Services, LLC (BWCS) declared that the last of the seven conversion lines from the two plants in Paducah (four lines) and Portsmouth (three lines) were fully operational. While conducting the hot functional testing and commencing full operations, PPPO identified CONOPS issues at both plants. BWCS acknowledged these concerns and developed a CONOPS improvement plan to ensure safe and efficient operations to meet the project schedule milestone of all seven lines at steady-state commercial production by the end of September 2012. A formal PPPO assessment of the BWCS CONOPS program is scheduled for the fourth quarter of fiscal year (FY) 2012. At the Portsmouth and Paducah Plants, direction and oversight for the operation of the DUF6 conversion plants is provided by the DUF6 Federal Project Director located in the PPPO office in Lexington, Kentucky and reporting to the DOE Office of Environmental Management. BWCS is the primary contractor responsible for project management and operation of the DUF6 conversion plants, which are categorized as nonreactor nuclear facility hazard category 3 pursuant to DOE-STD-1027, Hazard Categorization and Accident Analysis Techniques for Compliance with DOE Order 5480.23, Nuclear Safety Analysis Reports. The mission of the DUF6 conversion plants is to process the contents of more than 63,000 cylinders of DUF6 into a more stable form. A typical cylinder holds up to 14 tons of material. A cylinder is loaded into an autoclave where the DUF6 is heated and becomes a gas. The gas enters a conversion unit, where steam and hydrogen react with the DUF6 to form uranium oxide and hydrofluoric acid. The oxide powder is transferred to recycled cylinders; the offgas containing hydrofluoric acid is condensed with the liquid draining into a HF receiver tank and the gas going through a deionized water scrubbing system, which is also routed to the HF receiver tank. The plants have transitioned from a testing phase to a phased restart operational phase.
1
3.0 SCOPE The independent assessment focused on the requirements that support the key elements in DOE Order 226.1B, Implementation of Department of Energy Oversight Policy, with emphasis on the project office’s oversight of the identification, evaluation, analysis, transmittal, and closure of issues related to CONOPS at both DUF6 conversion plants. The contractor is transitioning from DOE Order 5480.19, Conduct of Operation Requirements for DOE Facilities, to DOE Order 422.1, Conduct of Operations. The tailored review of CONOPS oversight included an assessment of the site’s oversight processes and the processes for scheduling and conducting oversight activities; review of assessment documentation; shadowing and observation of project office personnel conducting operational awareness/surveillances; and interviews with personnel.
4.0 RESULTS Objective 1: Evaluate the DOE oversight program for determining the effectiveness of the contractor’s performance through the results of operational awareness activities and assessments of facilities and operations, including event investigation and self-assessment programs. The management systems employed by the DUF6 Project are described in the PPPO Management Plan, which serves as the Functions, Responsibilities, and Authorities (FRA) Document, the DOE Safety Management System Description, and the Project Management Plan. The Management Plan assigns responsibility for the day-to-day assessment of program operations to the PPPO manager, who flows that responsibility down to the PPPO oversight groups to verify that the contractor’s management system is controlling CONOPS. The Management Plan also recognizes the use of Integrated Project Teams (IPTs) at Portsmouth and Paducah. The Nuclear Safety Oversight Lead (NSOL) is identified as the functional lead for the Facility Representative (FR) program and for CONOPS. The line manager for the DUF6 Project is the Federal Project Director (FPD), with deputy FPDs located at each site. The Management Plan specifically identifies FRs as the primary onsite points of contact between PPPO and the site contractor, with responsibilities for ensuring that the CONOPS program is implemented and verifying the resolution of identified deficiencies and acceptable closure of issues. The Management Plan describes the technical management of the sites, including expectations for the CONOPS program. Safety management functions, responsibilities and authorities are listed in Appendix C of the Management Plan, but the FRA has not been updated in five years. (See Finding F-1.) PPPO-M-226.1-2, Oversight Program Plan, describes the process for oversight, including the types of oversight, the program areas to be reviewed, periodicity, review criteria, and qualifications of personnel performing reviews. It includes both oversight of the contractor and self-assessments of PPPO programs. Roles and responsibilities are assigned, with the PPPO manager responsible for approving the annual assessment schedule and the NSOL responsible for the oversight program, including developing the annual assessment schedule. The oversight program plan identifies eight key focus areas of project execution, including integrated safety management system (ISMS), quality assurance (QA), project management, facility operations, facility safety, radiation protection, safeguards and security, and operational readiness. It also addresses the use of review criteria, types of oversight methods, and issues and corrective actions. However, the oversight program plan does not reflect current directives. (See Finding F-1.) PPPO-M-414.1-5E, Quality Assurance Program Plan, serves as the QA plan for PPPO and shows how the applicable requirements from DOE Order 414.1C and ASME Nuclear Quality Assurance (NQA)-1 are addressed. The requirement for management assessments and independent assessments (both internal and external) includes the development of an annual integrated assessment schedule, including self-
2
assessments. The QA program plan adequately describes the QA processes, but is not current (e.g., it does not incorporate DOE Order 414.1D, which was issued in 2011). (See Finding F-1.) PPPO-M-414.1-2, Assessment and Surveillance Process, defines the assessment and surveillance processes and assigns responsibilities. Requirements are provided for the development of the annual integrated assessment/surveillance schedule and the planning and performance of assessments and surveillances. The assessment team leader is expected to be a certified NQA-1 lead auditor. An assessment plan should be prepared and approved, including the use of Criteria, Review, and Approach Documents (surveillances can be performed using checklists). The process for follow-up on assessments/surveillances is also included. The assessment and surveillance document provides thorough and detailed direction for the planning and performance of reviews. However, the section on developing the integrated assessment/surveillance schedule does not address the need to consider the effectiveness of contractor assurance systems, the hazards at the site/activity, and the degree of risk, with additional emphasis on potentially high-consequence activities. (See Opportunity for Improvement OFI-1.) The FY 2012 annual integrated assessment schedule for PPPO, as well as the DUF6 conversion project assessment schedules include scheduled assessments in a number of functional areas, including CONOPS. To ensure that the assessment schedule includes all required assessments, the QA contractor responsible for coordinating the integrated assessment schedule has developed a table of DOE assessment drivers, including the requirement and frequency. DOE IPT self-assessments are scheduled as determined by the DUF6 FPD. Recent self-assessments include one in FY 2011 addressing nuclear safety culture, one in FY 2010 on the Field Inspection Report (FIR) process, and one in FY 2012 addressing the DOE operating experience program. Independent Oversight reviewed the DUF6 PPPO IPT assessment report on nuclear safety culture. Although the report states that it is a self-assessment, the checklists, lines of inquiry, documents, interviews, and recommendations are all related to the contractor, BWCS. The self-assessment performed for the FIR process was found to have an appropriate scope, and it identified two areas for improvement: the tracking of issues identified in FIRs, and the response time by the contractor. The small sample of assessment reports that were reviewed showed that assessments were led by an NQA-1 certified lead auditor, and the reports included the scope, the review team, checklist/lines of inquiry, and recommendations and observations. The assessment reports were formally transmitted from the FPD to the contractor. In addition to the oversight management plans issued by PPPO, the DUF6 Project issued a projectspecific oversight plan addressing the performance of surveillances. Revision 3 of PMP-5-08, Oversight Plan, was recently issued to reflect the change in project status to operations and maintenance. The plan specifically includes the area of CONOPS, assigns responsibilities, and describes surveillance planning, performance, and follow-up. The results of the surveillances are documented in FIRs, which are submitted monthly to the contractor. The primary responsibility for performing CONOPS oversight resides with the FRs. PPPO-1063, Facility Representative Program Plan, defines the roles and responsibilities of FRs and addresses the selection, training, and qualification requirements for FRs. It also discusses the responsibilities of the NSOL, who is the functional manager for the FR program. A management self-assessment of the PPPO FR program in March 2010 concluded that the PPPO FR program was conducted consistent with DOE-STD-10632006. The self-assessment was thorough and well-documented. The FRs assigned to the DUF6 conversion project have completed the qualification process and have extensive experience in overseeing nuclear operations. In addition to the oversight provided by the FRs, a cadre of highly qualified and experienced technical support contractors is assigned to perform operational oversight. PPPO-M-420.1-3, Safety Systems Oversight Program Plan, describes the roles and responsibilities for the PPPO safety
3
system oversight (SSO) engineers, including assessing their assigned safety systems. PPPO has one SSO engineer who is currently assigned to Portsmouth but provides support to both plants, and the NSOL at Paducah also provides SSO support. The program plans for the FRs and SSOs adequately define the programs, although the FR program plan references an outdated version of DOE-STD-1063. (See Finding F-1.) The FRs and IPT have identified issues through their assessment and operational awareness activities and appropriately document those issues in assessment reports or FIRs. Independent Oversight reviewed several FIRs and found that they covered appropriate topics (e.g., log-keeping, worker involvement, operator aids, crane inspections, maintenance, combustible loading), were technically accurate, identified issues, and were transmitted to the contractor by the FPD. In particular, concerns about the CONOPS program were first identified in a Portsmouth FIR. Prior to the contractor’s planned CONOPS assessment, the PPPO conducted inspections in the areas of control of equipment and system status, technical procedures, required reading, lockout/tagout, and shift turnover and assumption of responsibilities. These FIRs were initially transmitted to the contractor as Information Only, but due to the number and nature of the findings, the FIRs were resubmitted with the request that BWCS prepare corrective action plans and modify the existing CONOPS improvement plan to address the identified issues. Numerous FIRs have also been generated with respect to the condition reporting program, resulting in development of a condition reporting process improvement plan. Concerns about the self-assessment aspect of a robust CONOPS program were identified to the contractor during the review of the CONOPS improvement plan. These concerns included the lack of an internal program/process for self-improvement. Additionally, the contractor’s performance metrics show that not all scheduled management assessments have been conducted. Independent Oversight observed the FR performing a walkdown of the process offgas in-line oxygen sensor and his oversight of a sensor recently declared out of service because the instrument was not successfully recalibrated upon receipt of a high oxygen value. The FR recounted the sequence of events of the startup of the conversion line, the receipt of the high oxygen alarm, and the attempt to recalibrate the sensor. After several days of conversion line operation, an operator discovered that the water seal leg was dry. When the seal leg water level was restored to the requirement of greater than or equal to 48 inches as stated in the startup section of the operating procedure, the oxygen value returned to normal. The FR reviewed the appropriate documentation, such as the maintenance package, work order log, facility manager and shift supervisor logs, and the operations manager’s standing order. The FR documented the operations and maintenance issue in the weekly report and determined that this event was of high significance. The FR weekly report documented that no condition report was issued following the discovery. The FR’s knowledge and review of the event were satisfactory. The FR and IPT staff are very knowledgeable of the DUF6 conversion project operations, processes, and safety systems. Regarding event investigation, the FRs attend fact-finding meetings and review occurrence reports, providing input as needed. The weekly reports include scheduled and planned activities; a summary of activities observed/completed; and deficiencies, issues, and concerns. The weekly reports are an excellent snapshot of operational activities and concerns. A weekly meeting of the FRs, SSO, FPD, NSOL, and other PPPO subject matter experts identifies items of interest to the FR team and provides an opportunity to highlight concerns to management. The Independent Oversight team observed the BWCS Monthly Project Review meeting. Based on feedback from the DOE FPD, the BWCS management team focuses on a subset of high-level metrics covering environment, safety, and health; training; engineering; operations; maintenance; and conversion production. The charts provide recent trends and goals. Interaction between the DOE FPD and BWCS management was good, as indicated by the request for further information behind several specific
4
metrics’ performance and clarification of definitions, such as system health reports. The charts did not specifically identify which metrics were associated with the CONOPS focus areas. From interviews, personnel gave examples of weaknesses in CONOPS areas, such as communication, shift turnover, procedure compliance, system knowledge, and log keeping. However, the interviewed personnel did not specifically relate the weaknesses to CONOPS focus areas, and specific metrics that would drive improvement in these areas were not identified. For example, if shift turnover, log keeping, and use of human performance improvement tools were current CONOPS issues, then metrics that count the positive characteristics could provide the data for management to determine whether these issues need increased attention or should be removed from the list. Overdue training and overdue preventive maintenance are two performance metrics that have been showing improvement over the past several months. The current human performance metric is defined as the number of employee concerns opened; however that metric does not consider the observations of the use of human performance tools such as quality of pre-job briefs, procedure use and adherence, touch STAR (Stop Think Act Review), and post-job reviews. PPPO and BWCS have acknowledged that effective resolution of reported problems is a weakness. BWCS has decreased the backlog of legacy Condition Reports (CRs) by 69% in the last year since operating the facilities. However, in the March Monthly Project Report, approximately 60% of the 178 open CRs were overdue. Also, BWCS has not reviewed all open CRs for validity, establish priorities, and assign due dates and responsible departments for completion. PPPO has formally communicated to BWCS that BWCS has not entered all operationally significant events and conditions into the Condition Reporting system for analysis, tracking, and trending. The failure to issue a CR on the operational and maintenance issue with the oxygen sensor is one recent example. BWCS has responded that clearer expectations and guidance have been prepared and will be incorporated in the next revision of the CR system. The PPPO personnel have demonstrated their knowledge of good DOE and commercial nuclear practices in the performance of their operational awareness surveillances and assessments. Independent Oversight provided PPPO personnel with several Institute of Nuclear Power Operations (INPO) publications that could be considered as part of continuous improvement initiatives. INPO 09-011, Achieving Excellence in Performance Improvement, lists typical behaviors representative of performance improvement and examples of performance metrics associated with the key success factors. INPO 01-005, Indicators of Changing Performance, and INPO 05-005, Guidelines for Performance Improvement at Nuclear Power Stations, are other potentially useful documents provided by the Independent Oversight team. Data already collected from the nuclear safety culture survey, CONOPS improvement plan, DOE and BWCS assessments, and senior supervisory watch reports could be used to develop additional metrics that would be more focused on driving performance toward the desired results. (See Opportunity for Improvement OFI-2.) The combination of PPPO and DUF6 conversion project procedures and documents that govern the site’s oversight process, and the process for scheduling and conducting activities, provide an acceptable basis for establishing an oversight program at the DUF6 conversion plants. Effective oversight is being provided by the FRs, technical support contractors, NSOL, SSO, and the IPT. Concerns are appropriately identified through assessment and operational awareness activities, and in most cases they are formally transmitted to the contractor for resolution. The management systems in place for oversight are appropriate to promote improvement in the contractor’s performance, given appropriate emphasis on evaluating and correcting the issues.
5
Objective 2: Evaluate the project office’s issues management processes for: a) categorizing findings based on risk and priority; b) ensuring that relevant management findings are effectively communicated to the contractors; and c) ensuring that problems are evaluated and corrected on a timely basis. Both PPPO-M-414.1-5E, Quality Assurance Program Plan, and PPPO-M-414.1-1, Corrective Action Program, use the all-inclusive term “condition adverse to quality” (CAQ) for problems, failures, malfunctions, deficiencies, defective items, and nonconformances. The corrective action program (CAP) document additionally states that CAQs may be reported as findings, observations, significant CAQs, problem reports, judgments of need, or nonconformances. A significant CAQ is defined as a quality that, if left uncorrected, could have a serious effect on safety, the environment, or operability. No further process is identified to categorize the findings based on risk and priority. (See Opportunity for Improvement OFI- 3.) For issues categorized as high-significance findings (significant CAQs), the CAP procedure and the surveillance and assessments procedure require that an extent-of-condition and root cause analysis be performed, and that a cause code be assigned for all identified causes. They further require that corrective actions (CAs) should address all aspects of each identified causal factor and should provide both remedial actions to address the finding and preventive actions to prevent recurrence. A similar process is followed for CAQs, using a graded approach for the level of causal analysis. Section 5.7 of the procedure addresses verification of CAs, including effectiveness verifications. Although there is some evidence that effectiveness reviews are included in follow-up assessments, interviews with the staff indicated that these reviews primarily verify that the CAs have been completed, not that they are effective. (See Opportunity for Improvement OFI-4.) Tracking of CAs resulting from management assessments is addressed in the CAP procedure, which states that QA personnel enter the CAs into the QA tracking system. Tracking of CAs resulting from surveillances is not specifically addressed in the procedure on the assessment and surveillance process; however, the DUF6 Project oversight plan for operations and maintenance states that verification of CAs will be tracked as part of the DUF6 Project Letter Transmittal Tracking. It further states that tracking and verification of issues are the responsibility of the FIR author. Interviews with PPPO personnel indicate that some activities are tracked in the management tracking system, but most of the staff do not use the system. No overall issues management tracking system is used at PPPO. (See Opportunity for Improvement OFI-5.) The oversight process, as discussed in Objective 1, has identified relevant management findings. The issues identified as the result of management assessments are formally transmitted to the contractor, and the CAs are tracked in the QA tracking system. Surveillances and operational awareness activities are documented on FIRs, which are provided to the contractor electronically. The IPT submits a monthly letter to the contractor containing all FIRs, with an action response for “marginal” or “unsatisfactory” FIRs due within 15 days of receipt of the letter. The monthly letters are transmitted to the contractor by the FPD, who is a contracting officer’s representative. Independent Oversight reviewed a sample of the transmittal letters for assessment reports and FIRs and found them to effectively communicate the issues to the contractor. However, in the absence of a well-defined process for categorizing findings based on risk and priority, the significance of the issue is not fully communicated to the contractor. One example of the FR’s oversight activities involved observation of the calibration of process offgas inline oxygen sensors. The contractor identified the need to verify the seal leg water level once per shift and issued Standing Order Number SO-OPS-12-005. The standing order required the level of the seal leg water to be verified as greater than or equal to 48 centimeters. However, the units should have been in inches; i.e., the level should be verified to be greater than or equal to 48 inches. The FR noted this
6
discrepancy to the operations manager, who revised the standing order. While these actions addressed the immediate issue, this issue was not formally transmitted to the contractor. Further, it is indicative of a broader issue: The company that provided the process conversion equipment installed metric units on the equipment, but the balance-of-plant contractor used the English system of measurement. (See Opportunity for Improvement OFI-6.) Regarding the evaluation and correction of problems on a timely basis, the PPPO DUF6 IPT has identified and formally transmitted to the contractor concerns about the CONOPS program and the condition reporting program. The contractor’s initial response to these concerns was less than adequate, resulting in additional follow-up and rework. The first direction to the contractor to develop a CONOPS improvement plan was transmitted on September 26, 2011, with a due date within 15 days. BWCS replied on October 26, 2011, with a CONOPS improvement plan that lacked specificity and did not result in improved performance, as demonstrated by the number of findings identified during the focused CONOPS inspections conducted by PPPO in January and February 2012. A follow-up letter was submitted to BWCS on February 29, 2012, requesting that the existing CONOPS improvement plan be modified to address the additional issues, no later than 15 days from receipt of the letter. On March 20, 2012, BWCS provided a status report. Similarly, FIRs were formally transmitted to the contractor on October 25, 2011, identifying concerns about the condition reporting system. On December 21, 2011, the contractor committed to implementing a condition reporting process improvement plan. On January 19, 2012, the condition reporting process improvement plan was formally transmitted to PPPO; it consisted of a project schedule chart, with no supporting documentation. PPPO continued to identify concerns about the condition reporting system in a February 9, 2012, letter to the contractor, with a 15 day response request. The contractor provided a response to this letter on March 16, 2012, but still had not presented the condition reporting improvement plan to the project office. This sequence of events demonstrates a concern about the contractor’s ability to evaluate and correct problems on a timely basis. (See Opportunity for Improvement OFI-2.)
5.0 CONCLUSIONS The combination of PPPO and DUF6 Project procedures and documents that govern the site’s oversight process, and the process for scheduling and conducting activities, provide an acceptable basis for establishing an oversight program at the DUF6 conversion plants. Effective oversight is provided by the FRs, technical support contractors, NSOL, SSO, and the IPT. Concerns are appropriately identified through the assessment and operational awareness activities, and in most cases they are formally transmitted to the contractor for resolution. The management systems in place for oversight are appropriately designed to result in improved contractor performance, given appropriate emphasis on evaluating and correcting the issues. However, one finding was identified related to the need to periodically review and update management documents; several of them referenced outdated orders and standards. In addition, the assessment planning process does not sufficiently consider the effectiveness of contractor assurance systems, the hazards at the site/activity, and the degree of risk (e.g., giving additional emphasis to potentially highconsequence activities). The PPPO/DUF6 processes for issues management provide for categorization of CAQs as findings, observations, significant CAQs, problem reports, judgments of need, or nonconformances. However, there is no further categorization of findings based on risk and priority. The processes for communicating management issues to the contractor are effective; the results of assessment reports are formally transmitted to the contractor, and the FIRs are formally transmitted to the contractor from the FPD on a monthly basis. PPPO has recognized a concern that the contractor’s initial response to issues concerning
7
the CONOPS program and the condition reporting program did not contain adequate detail and have not resulted in improvements to the programs, The FPD has communicated to the contractor the need to improve in these areas. Opportunities to improve the issues management process include the assessment planning process, the issue categorization process, the effectiveness review process, and the management tracking systems.
6.0 FINDINGS AND OPPORTUNITIES FOR IMPROVEMENT During the review, Independent Oversight identified several issues, which were characterized in accordance with the PPPO procedure for assessment and surveillances and are annotated by type (i.e., a finding or observation). The PPPO issues management process defines a CAQ as an “an all-inclusive term used in reference to any of the following: problems, failures, malfunctions, deficiencies, defective item, nonconformance. These may be reported as findings, observations, significant CAQ, or other similar terms.” Observations closely approximate opportunities for improvement, which according to Independent Oversight protocols “are suggestions offered by the Independent Oversight appraisal team that may assist line management in identifying options and potential solutions to various issues identified during the conduct of the appraisal.” The finding and opportunities for improvement are summarized below and are provided to PPPO/DUF6 for evaluation and follow-up in accordance with PPPO procedures and processes. Independent Oversight identified one finding and six opportunities for improvement (observations). Findings F-1: Many key PPPO process procedures include outdated references to DOE orders and standards and the Functions, Responsibilities and Authorities list has not been revised as required in five years. Opportunities for Improvement OFI-1: Strengthen the process for developing the annual integrated assessment schedule to ensure appropriate focus on higher-risk areas. OFI-2: Investigate the use of additional measures (e.g., award fee, performance metrics, specific direction) to reinforce the expectation that the evaluation of CONOPS problems be performed in a high quality manner resulting in improvement, and that problems be corrected expeditiously. OFI-3: Develop a well documented issues categorization process that prioritizes findings based on risk and priority and improves communications to the contractor of the significance of the issues. OFI-4: Establish an effectiveness review process that includes effectiveness reviews in the integrated assessment schedule and incorporates training of staff. OFI-5: Investigate the use of an issues management tracking system, such as Orion or Pegasus. OFI-6: Direct BWCS to conduct a human factors evaluation of the use of both English and metric systems.
8
7.0 FOLLOW-UP ITEMS As part of its regular oversight activities, Independent Oversight will follow the closure of the finding identified above and monitor the execution of the CONOPS program at the DUF6 conversion plants.
9
Appendix A Supplemental Information Dates of Review Onsite Review: March 12-16, 2012
Office of Health, Safety and Security Management Glenn S. Podonsky, Chief Health, Safety and Security Officer William A. Eckroade, Principal Deputy Chief for Mission Support Operations John S. Boulden III, Director, Office of Enforcement and Oversight Thomas R. Staker, Deputy Director for Oversight William E. Miller, Deputy Director, Office of Safety and Emergency Management Evaluations Quality Review Board William Eckroade John Boulden Thomas Staker William Miller Michael Kilpatrick George Armstrong Robert Nelson Independent Oversight Site Lead for Portsmouth/Paducah Project Office Joseph Drago Independent Oversight Reviewers Joseph Drago – Lead Terry Olberding
A-1
Appendix B Documents Reviewed, Interviews, and Observations Documents Reviewed • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • BWCS Paducah Facility Manager and Shift Supervisor Narrative Log BWCS Standing Order SO-OPS-12-005, Seal Leg Water Levels, March 13, 2012 BWCS Standing Order SO-OPS-12-005, Seal Leg Water Levels, March 13, 2012. Rev 1 BWCS-C-MNT-0614, Functional Testing of Conversion Unit High Pressure, Condenser High Temperature, Off Gas High Pressure, and Oxygen Indicators, R1, 3/12/12 C-201109-008, Field Inspection Report, Paducah Post-Start Readiness Review Findings, 9/8/11 C-201109-10, Paducah Condition Report Closure Review – second quarter, 9/14/11 C-201109-11, Paducah Field Inspection Report, Condition Report Closure Review – third quarter, 9/12/11 Daily Communication and Teamwork Report, 3/13/12 Daily Communication and Teamwork Report, 3/14/12 DE-AC30-11CC40015, Contract exhibit 3 award fee-base section rating criteria worksheets DOE O 226.1B, Implementation of Department of Energy Oversight Policy, 4/25/11 DOE O 422.1, Conduct of Operations, 6/29/10 DOE O 5480.19, Conduct of Operation Requirements for DOE Facilities, 10/23/2001 DOE O 450.2, Integrated Safety Management, 4/25/11 DUF6 Conversion Project Assessment Schedule – Annual Look-A-Head, 4/15/11 DUF6 Conversion Project Assessment Schedule – Semi-Annual Look-A-Head, 12/30/11 DUF6 Conversion Project Assessments/Audits/Surveillance Schedule October 2009September 2010, 10/4/10 Email from Gregory Bazzell to Daniel Whitley, subject comments on two occurrence repots, 12/15/11 Email from Gregory Bazzell to Daniel Whitley, subject comments on occurrence report, 2/23/12 Email from Jim Kenney to Charles Armitage, subj: Modifications to CONOPS plan, 10/20/11 Email from Jim Kenney to Peter Burban, subj: CONOPS Surveillance plan for next 8 weeks, 1/4/12 Email from Michelle Reichert to Haug, et al, subj: Rollup of Safety Culture Assessment Results, 2/8/12 Email Robinson to Bazzell, Rollup of Safety Culture Assessment Results, March 15, 2012. Facility Specific Qualification Card, Portsmouth Conversion Facilities, 3/23/10 Facility Specific Qualification Card, Paducah Conversion Facilities, 8/27/10 FR/SSO Weekly Meeting Summary, 2/22/2012 FR/SSO Weekly Meeting Summary, 2/29/2012 Fy2012 PPPO Assessment Schedule Handout, DUF6 Monthly Project Review, 3/14/12 Handout, DUF6 Project Performance Metrics, FY 2012 through 2/12 Letter from Dials to Zimmerman, subj: Combined Integrated Safety Management System (ISMS) and Quality Assurance Declaration – FY 2011, 11/30/11
B-1
• • • • • • • • • • • • • • • • • • • • • • • • • • • • •
Letter from Dials to Zimmerman, subj: Conduct of Operations Improvement Plan, 10/26/11 Letter from Dials to Zimmerman, subj: DUF6 Conduct of Operations Improvement Plan, 3/20/2012 Letter from Dials to Zimmerman, subj: Improved Use of the Condition Report (CR) System for Operations and Maintenance Issues, 3/16/2012 Letter from Dials to Zimmerman, subj: Response to “Forwarding of Unsatisfactory December 2011 Field Inspection Report” letter, 1/19/12 Letter from Dials to Zimmerman, subj: Response to “Insufficient Response to Marginal and Unsatisfactory September 2011Field Inspection Report Letter, dated 12/21/11 Letter from Reichert to Zimmerman, subj: Safety Culture Assessment for the Piketon and Paducah DUF6 Facilities, 12/2/11 PADU-10-IA-056, Management Assessment of PPPO Facility Representative Program, 3/10 Paducah FR Weekly Report, 2/20/12 – 2/26/12 Paducah FR Weekly Report, 2/27/12 – 3/2/12 Paducah FR Weekly Report, 3/5/12 – 3/11/12 Paducah Report for Week Ending 2/24/12 Paducah Report for Week Ending 3/2/12 Paducah Report for Week Ending 3/9/12 PMP-5-08, DUF6 Conversion Project Oversight Plan – Operations and Maintenance, R3, 3/5/12 PMP-5-08, DUF6 Conversion Project Oversight Plan – Transition to Operations, R2, 8/11/10 Portsmouth Weekly Report 2/20/12 – 2/26/12 Portsmouth Weekly Report 2/27/12 – 3/4/12 Portsmouth Weekly Report 3/5/12 – 3/11/12 PPPO Functions, Responsibilities and Authorities, 1/25/2007 PPPO-01-1130-11, letter from Zimmerman to Dials, subj: Forwarding of Unsatisfactory September 2011 Piketon Field Inspection Report, 9/26/11 PPPO-01-1133-11, letter from Zimmerman to Dials, subj: Forwarding of September 2011 Field Inspection Reports and Surveillance Reports, 9/29/11 PPPO-01-1136-11, letter from Zimmerman to Dials, subj: Nuclear Safety Culture Management Assessment Report #PADU-11-MA-100329, R1, 9/29/11 PPPO-01-1139, letter from Zimmerman to Dials, subj: Forwarding of Unsatisfactory September 2011 Field Inspection Reports, 10/6/11 PPPO-01-1149-12, letter from Zimmerman to Dials, subj: Insufficient Response to Marginal and Unsatisfactory September 2011 Field Inspection Report Letter, 10/25/11 PPPO-01-1161-12, Letter from Zimmerman to Dials; subj: Qualification and Training Program Management Assessment Report PPPO-11-MA-100331, R2, 11/16/11 PPPO-01-1169-12, Letter from Zimmerman to Dials, subj: Forwarding of October 2011 Field Inspection Report, 11/21/11 PPPO-01-1171-12, letter from Zimmerman to Dials, subj: Forwarding of November 2011 Field Inspection Reports, 12/14/11 PPPO-01-1178-12, letter from Zimmerman to Dials, subj: Forwarding of Unsatisfactory December 2011 Field Inspection Report, 12/20/11 PPPO-01-1179-12, Letter from Zimmerman to Dials, 12/22/11; subj: Authorized Release of Aqueous Hydrofluoric Acid Program 2nd Quarterly Assessment Report #PADU-11MA-100344, R1
B-2
• • • • • • • • • • • • • • • • • • • • •
PPPO-01-1187-12, Letter from Zimmerman to Dials; subj: Integrated Project Team Oversight Assessment Schedule and Near-Term Planned Assessments, 1/11/12 PPPO-01-1189-12, letter from Zimmerman to Dials, subj: Forwarding of Marginal December 2011 Field Inspection Reports, 1/11/12 PPPO-01-1388916-12, Letter from Zimmerman to Dials, subj: Forwarding of January 2012 Field Inspection Reports, 2/8/12 PPPO-01-1390541-12, letter from Zimmerman to Dials, subj: Improved Use of the Condition Report System for Operations and Maintenance Issues, 2/9/12 PPPO-01-1390541-12, letter from Zimmerman to Dials, subj: Improved Use of the Condition Report System for Operations and Maintenance Issues, 2/9/12 PPPO-01-1413751-12, Letter from Zimmerman to Dials, subj: Forwarding of Informational January and February 2012 Field Inspection Reports, 2/29/12 PPPO-01-981250-10B, memo from Murphie to file, subj
UF6 Facility Representative Qualifications, 8/12/10 PPPO-1063, Facility Representative Program Plan, R1, 3/10 PPPO-M-226.1-2, Oversight Program Plan, R1, 3/10 PPPO-M-413.1-1, Management Plan, R3, 3/10 PPPO-M-414.1-1, Corrective Action Program, R1, 3/2010 PPPO-M-414.1-2, Assessment and Surveillance Process, R1 3/2010 PPPO-M-414.1-5E, Quality Assurance Program Plan, R5, 2/2010 PPPO-M-420.1-3, Safety Systems Oversight Program Plan, R0, 2009 Report, DOE Assessment Drivers Safety System Oversight Qualification Card, 2/25/10 Support Contractor Qualification Card, 3/7/11 Various ORPS reports X-201009-24, Field Inspection Report, Portsmouth, Self-Assessment of FIRs, 9/10-20/10 X-201109-011, Portsmouth, Field Inspection Report, Portsmouth Post-Start Readiness Review Findings, 9/2/11 X-201109-12, Piketon Field Inspection Report, Condition Reporting, 9/12/11
Interviews Nuclear Safety Oversight Lead DOE Facility Representative for DUF6 Paducah DOE Facility Representative for DUF6 Portsmouth BWCS Paducah Plant Manager BWCS Portsmouth Plant Manager Federal Project Director Deputy Federal Project Director for Paducah Deputy Federal Project Director for Portsmouth DOE Safety System Oversight Engineer PRC Support Contractor Lead PRC Support Contractor QA PRC Support Contractors Paducah PRC Support Contractors Portsmouth
B-3
Observations DUF6 Paducah plant tour Monthly Project Status meeting Daily Communication and Teamwork meeting Shadow of FR walkthrough
B-4
doc_785864016.pdf
An operational definition, also called functional definition,[1][2] is a result of the process of operationalization and is used to define something (e.g. a variable, term, or object) in terms of a process (or set of validation tests) needed to determine its existence, duration, and quantity.
Independent Oversight Assessment of the Portsmouth/Paducah Project Office Conduct of Operations Oversight of the Depleted Uranium Hexafluoride Conversion Plants
May 2012 Office of Safety and Emergency Management Evaluations Office of Enforcement and Oversight Office of Health, Safety and Security U.S. Department of Energy
Table of Contents
1.0 Purpose................................................................................................................................................... 1 2.0 Background ............................................................................................................................................ 1 3.0 Scope ...................................................................................................................................................... 2 4.0 Results ................................................................................................................................................... 2 5.0 Conclusions ........................................................................................................................................... 7 6.0 Findings and Opportunities for Improvement........................................................................................ 8 7.0 Follow-up Items ..................................................................................................................................... 9 Appendix A: Supplemental Information .................................................................................................. A-1 Appendix B: Documents Reviewed, Interviews, and Observations ........................................................ B-1
i
Acronyms
BWCS CA CAP CAQ CR CONOPS DOE DUF6 FIR FPD FR FRA FY HSS INPO IPT ISMS NQA NSOL PPPO QA SSO
Babcock & Wilcox Conversion Service LLC Corrective Action Corrective Action Program Condition Adverse to Quality Condition Report Conduct of Operations U.S. Department of Energy Depleted Uranium Hexafluoride Field Inspection Report Federal Project Director Facility Representative Functions, Responsibilities and Authorities Fiscal Year Office of Health, Safety and Security Institute of Nuclear Power Operations Integrated Project Team Integrated Safety Management System Nuclear Quality Assurance Nuclear Safety Oversight Lead Portsmouth/Paducah Project Office Quality Assurance Safety System Oversight
ii
Independent Oversight Assessment of the Portsmouth/Paducah Project Office Conduct of Operations Oversight of the Depleted Uranium Hexafluoride Conversion Plants
1.0 PURPOSE This report documents the U.S. Department of Energy (DOE) Office of Enforcement and Oversight (Independent Oversight), within the Office of Health, Safety and Security’s (HSS) independent assessment of the Portsmouth/Paducah Project Office (PPPO) oversight of conduct of operations (CONOPS) at the depleted uranium hexafluoride (DUF6) conversion plants. The Independent Oversight assessment focused on PPPO’s oversight of the identification, evaluation, analysis, transmittal, and closure of issues related to CONOPS at the DUF6 plants at the Portsmouth Gaseous Diffusion Plant, Piketon, Ohio, and the Paducah Gaseous Diffusion Plant, Paducah, Kentucky. The onsite portion of the review was performed March 12-16, 2012, by the HSS Office of Safety and Emergency Management Evaluations. The objectives of the Independent Oversight assessment were to evaluate: (1) the PPPO oversight program’s effectiveness in improving the contractor’s performance; and (2) the project office’s issues management processes.
2.0 BACKGROUND In 2010, DOE authorized hot functional testing of the two DUF6 conversion plants. On September 30, 2011, Babcock & Wilcox Conversion Services, LLC (BWCS) declared that the last of the seven conversion lines from the two plants in Paducah (four lines) and Portsmouth (three lines) were fully operational. While conducting the hot functional testing and commencing full operations, PPPO identified CONOPS issues at both plants. BWCS acknowledged these concerns and developed a CONOPS improvement plan to ensure safe and efficient operations to meet the project schedule milestone of all seven lines at steady-state commercial production by the end of September 2012. A formal PPPO assessment of the BWCS CONOPS program is scheduled for the fourth quarter of fiscal year (FY) 2012. At the Portsmouth and Paducah Plants, direction and oversight for the operation of the DUF6 conversion plants is provided by the DUF6 Federal Project Director located in the PPPO office in Lexington, Kentucky and reporting to the DOE Office of Environmental Management. BWCS is the primary contractor responsible for project management and operation of the DUF6 conversion plants, which are categorized as nonreactor nuclear facility hazard category 3 pursuant to DOE-STD-1027, Hazard Categorization and Accident Analysis Techniques for Compliance with DOE Order 5480.23, Nuclear Safety Analysis Reports. The mission of the DUF6 conversion plants is to process the contents of more than 63,000 cylinders of DUF6 into a more stable form. A typical cylinder holds up to 14 tons of material. A cylinder is loaded into an autoclave where the DUF6 is heated and becomes a gas. The gas enters a conversion unit, where steam and hydrogen react with the DUF6 to form uranium oxide and hydrofluoric acid. The oxide powder is transferred to recycled cylinders; the offgas containing hydrofluoric acid is condensed with the liquid draining into a HF receiver tank and the gas going through a deionized water scrubbing system, which is also routed to the HF receiver tank. The plants have transitioned from a testing phase to a phased restart operational phase.
1
3.0 SCOPE The independent assessment focused on the requirements that support the key elements in DOE Order 226.1B, Implementation of Department of Energy Oversight Policy, with emphasis on the project office’s oversight of the identification, evaluation, analysis, transmittal, and closure of issues related to CONOPS at both DUF6 conversion plants. The contractor is transitioning from DOE Order 5480.19, Conduct of Operation Requirements for DOE Facilities, to DOE Order 422.1, Conduct of Operations. The tailored review of CONOPS oversight included an assessment of the site’s oversight processes and the processes for scheduling and conducting oversight activities; review of assessment documentation; shadowing and observation of project office personnel conducting operational awareness/surveillances; and interviews with personnel.
4.0 RESULTS Objective 1: Evaluate the DOE oversight program for determining the effectiveness of the contractor’s performance through the results of operational awareness activities and assessments of facilities and operations, including event investigation and self-assessment programs. The management systems employed by the DUF6 Project are described in the PPPO Management Plan, which serves as the Functions, Responsibilities, and Authorities (FRA) Document, the DOE Safety Management System Description, and the Project Management Plan. The Management Plan assigns responsibility for the day-to-day assessment of program operations to the PPPO manager, who flows that responsibility down to the PPPO oversight groups to verify that the contractor’s management system is controlling CONOPS. The Management Plan also recognizes the use of Integrated Project Teams (IPTs) at Portsmouth and Paducah. The Nuclear Safety Oversight Lead (NSOL) is identified as the functional lead for the Facility Representative (FR) program and for CONOPS. The line manager for the DUF6 Project is the Federal Project Director (FPD), with deputy FPDs located at each site. The Management Plan specifically identifies FRs as the primary onsite points of contact between PPPO and the site contractor, with responsibilities for ensuring that the CONOPS program is implemented and verifying the resolution of identified deficiencies and acceptable closure of issues. The Management Plan describes the technical management of the sites, including expectations for the CONOPS program. Safety management functions, responsibilities and authorities are listed in Appendix C of the Management Plan, but the FRA has not been updated in five years. (See Finding F-1.) PPPO-M-226.1-2, Oversight Program Plan, describes the process for oversight, including the types of oversight, the program areas to be reviewed, periodicity, review criteria, and qualifications of personnel performing reviews. It includes both oversight of the contractor and self-assessments of PPPO programs. Roles and responsibilities are assigned, with the PPPO manager responsible for approving the annual assessment schedule and the NSOL responsible for the oversight program, including developing the annual assessment schedule. The oversight program plan identifies eight key focus areas of project execution, including integrated safety management system (ISMS), quality assurance (QA), project management, facility operations, facility safety, radiation protection, safeguards and security, and operational readiness. It also addresses the use of review criteria, types of oversight methods, and issues and corrective actions. However, the oversight program plan does not reflect current directives. (See Finding F-1.) PPPO-M-414.1-5E, Quality Assurance Program Plan, serves as the QA plan for PPPO and shows how the applicable requirements from DOE Order 414.1C and ASME Nuclear Quality Assurance (NQA)-1 are addressed. The requirement for management assessments and independent assessments (both internal and external) includes the development of an annual integrated assessment schedule, including self-
2
assessments. The QA program plan adequately describes the QA processes, but is not current (e.g., it does not incorporate DOE Order 414.1D, which was issued in 2011). (See Finding F-1.) PPPO-M-414.1-2, Assessment and Surveillance Process, defines the assessment and surveillance processes and assigns responsibilities. Requirements are provided for the development of the annual integrated assessment/surveillance schedule and the planning and performance of assessments and surveillances. The assessment team leader is expected to be a certified NQA-1 lead auditor. An assessment plan should be prepared and approved, including the use of Criteria, Review, and Approach Documents (surveillances can be performed using checklists). The process for follow-up on assessments/surveillances is also included. The assessment and surveillance document provides thorough and detailed direction for the planning and performance of reviews. However, the section on developing the integrated assessment/surveillance schedule does not address the need to consider the effectiveness of contractor assurance systems, the hazards at the site/activity, and the degree of risk, with additional emphasis on potentially high-consequence activities. (See Opportunity for Improvement OFI-1.) The FY 2012 annual integrated assessment schedule for PPPO, as well as the DUF6 conversion project assessment schedules include scheduled assessments in a number of functional areas, including CONOPS. To ensure that the assessment schedule includes all required assessments, the QA contractor responsible for coordinating the integrated assessment schedule has developed a table of DOE assessment drivers, including the requirement and frequency. DOE IPT self-assessments are scheduled as determined by the DUF6 FPD. Recent self-assessments include one in FY 2011 addressing nuclear safety culture, one in FY 2010 on the Field Inspection Report (FIR) process, and one in FY 2012 addressing the DOE operating experience program. Independent Oversight reviewed the DUF6 PPPO IPT assessment report on nuclear safety culture. Although the report states that it is a self-assessment, the checklists, lines of inquiry, documents, interviews, and recommendations are all related to the contractor, BWCS. The self-assessment performed for the FIR process was found to have an appropriate scope, and it identified two areas for improvement: the tracking of issues identified in FIRs, and the response time by the contractor. The small sample of assessment reports that were reviewed showed that assessments were led by an NQA-1 certified lead auditor, and the reports included the scope, the review team, checklist/lines of inquiry, and recommendations and observations. The assessment reports were formally transmitted from the FPD to the contractor. In addition to the oversight management plans issued by PPPO, the DUF6 Project issued a projectspecific oversight plan addressing the performance of surveillances. Revision 3 of PMP-5-08, Oversight Plan, was recently issued to reflect the change in project status to operations and maintenance. The plan specifically includes the area of CONOPS, assigns responsibilities, and describes surveillance planning, performance, and follow-up. The results of the surveillances are documented in FIRs, which are submitted monthly to the contractor. The primary responsibility for performing CONOPS oversight resides with the FRs. PPPO-1063, Facility Representative Program Plan, defines the roles and responsibilities of FRs and addresses the selection, training, and qualification requirements for FRs. It also discusses the responsibilities of the NSOL, who is the functional manager for the FR program. A management self-assessment of the PPPO FR program in March 2010 concluded that the PPPO FR program was conducted consistent with DOE-STD-10632006. The self-assessment was thorough and well-documented. The FRs assigned to the DUF6 conversion project have completed the qualification process and have extensive experience in overseeing nuclear operations. In addition to the oversight provided by the FRs, a cadre of highly qualified and experienced technical support contractors is assigned to perform operational oversight. PPPO-M-420.1-3, Safety Systems Oversight Program Plan, describes the roles and responsibilities for the PPPO safety
3
system oversight (SSO) engineers, including assessing their assigned safety systems. PPPO has one SSO engineer who is currently assigned to Portsmouth but provides support to both plants, and the NSOL at Paducah also provides SSO support. The program plans for the FRs and SSOs adequately define the programs, although the FR program plan references an outdated version of DOE-STD-1063. (See Finding F-1.) The FRs and IPT have identified issues through their assessment and operational awareness activities and appropriately document those issues in assessment reports or FIRs. Independent Oversight reviewed several FIRs and found that they covered appropriate topics (e.g., log-keeping, worker involvement, operator aids, crane inspections, maintenance, combustible loading), were technically accurate, identified issues, and were transmitted to the contractor by the FPD. In particular, concerns about the CONOPS program were first identified in a Portsmouth FIR. Prior to the contractor’s planned CONOPS assessment, the PPPO conducted inspections in the areas of control of equipment and system status, technical procedures, required reading, lockout/tagout, and shift turnover and assumption of responsibilities. These FIRs were initially transmitted to the contractor as Information Only, but due to the number and nature of the findings, the FIRs were resubmitted with the request that BWCS prepare corrective action plans and modify the existing CONOPS improvement plan to address the identified issues. Numerous FIRs have also been generated with respect to the condition reporting program, resulting in development of a condition reporting process improvement plan. Concerns about the self-assessment aspect of a robust CONOPS program were identified to the contractor during the review of the CONOPS improvement plan. These concerns included the lack of an internal program/process for self-improvement. Additionally, the contractor’s performance metrics show that not all scheduled management assessments have been conducted. Independent Oversight observed the FR performing a walkdown of the process offgas in-line oxygen sensor and his oversight of a sensor recently declared out of service because the instrument was not successfully recalibrated upon receipt of a high oxygen value. The FR recounted the sequence of events of the startup of the conversion line, the receipt of the high oxygen alarm, and the attempt to recalibrate the sensor. After several days of conversion line operation, an operator discovered that the water seal leg was dry. When the seal leg water level was restored to the requirement of greater than or equal to 48 inches as stated in the startup section of the operating procedure, the oxygen value returned to normal. The FR reviewed the appropriate documentation, such as the maintenance package, work order log, facility manager and shift supervisor logs, and the operations manager’s standing order. The FR documented the operations and maintenance issue in the weekly report and determined that this event was of high significance. The FR weekly report documented that no condition report was issued following the discovery. The FR’s knowledge and review of the event were satisfactory. The FR and IPT staff are very knowledgeable of the DUF6 conversion project operations, processes, and safety systems. Regarding event investigation, the FRs attend fact-finding meetings and review occurrence reports, providing input as needed. The weekly reports include scheduled and planned activities; a summary of activities observed/completed; and deficiencies, issues, and concerns. The weekly reports are an excellent snapshot of operational activities and concerns. A weekly meeting of the FRs, SSO, FPD, NSOL, and other PPPO subject matter experts identifies items of interest to the FR team and provides an opportunity to highlight concerns to management. The Independent Oversight team observed the BWCS Monthly Project Review meeting. Based on feedback from the DOE FPD, the BWCS management team focuses on a subset of high-level metrics covering environment, safety, and health; training; engineering; operations; maintenance; and conversion production. The charts provide recent trends and goals. Interaction between the DOE FPD and BWCS management was good, as indicated by the request for further information behind several specific
4
metrics’ performance and clarification of definitions, such as system health reports. The charts did not specifically identify which metrics were associated with the CONOPS focus areas. From interviews, personnel gave examples of weaknesses in CONOPS areas, such as communication, shift turnover, procedure compliance, system knowledge, and log keeping. However, the interviewed personnel did not specifically relate the weaknesses to CONOPS focus areas, and specific metrics that would drive improvement in these areas were not identified. For example, if shift turnover, log keeping, and use of human performance improvement tools were current CONOPS issues, then metrics that count the positive characteristics could provide the data for management to determine whether these issues need increased attention or should be removed from the list. Overdue training and overdue preventive maintenance are two performance metrics that have been showing improvement over the past several months. The current human performance metric is defined as the number of employee concerns opened; however that metric does not consider the observations of the use of human performance tools such as quality of pre-job briefs, procedure use and adherence, touch STAR (Stop Think Act Review), and post-job reviews. PPPO and BWCS have acknowledged that effective resolution of reported problems is a weakness. BWCS has decreased the backlog of legacy Condition Reports (CRs) by 69% in the last year since operating the facilities. However, in the March Monthly Project Report, approximately 60% of the 178 open CRs were overdue. Also, BWCS has not reviewed all open CRs for validity, establish priorities, and assign due dates and responsible departments for completion. PPPO has formally communicated to BWCS that BWCS has not entered all operationally significant events and conditions into the Condition Reporting system for analysis, tracking, and trending. The failure to issue a CR on the operational and maintenance issue with the oxygen sensor is one recent example. BWCS has responded that clearer expectations and guidance have been prepared and will be incorporated in the next revision of the CR system. The PPPO personnel have demonstrated their knowledge of good DOE and commercial nuclear practices in the performance of their operational awareness surveillances and assessments. Independent Oversight provided PPPO personnel with several Institute of Nuclear Power Operations (INPO) publications that could be considered as part of continuous improvement initiatives. INPO 09-011, Achieving Excellence in Performance Improvement, lists typical behaviors representative of performance improvement and examples of performance metrics associated with the key success factors. INPO 01-005, Indicators of Changing Performance, and INPO 05-005, Guidelines for Performance Improvement at Nuclear Power Stations, are other potentially useful documents provided by the Independent Oversight team. Data already collected from the nuclear safety culture survey, CONOPS improvement plan, DOE and BWCS assessments, and senior supervisory watch reports could be used to develop additional metrics that would be more focused on driving performance toward the desired results. (See Opportunity for Improvement OFI-2.) The combination of PPPO and DUF6 conversion project procedures and documents that govern the site’s oversight process, and the process for scheduling and conducting activities, provide an acceptable basis for establishing an oversight program at the DUF6 conversion plants. Effective oversight is being provided by the FRs, technical support contractors, NSOL, SSO, and the IPT. Concerns are appropriately identified through assessment and operational awareness activities, and in most cases they are formally transmitted to the contractor for resolution. The management systems in place for oversight are appropriate to promote improvement in the contractor’s performance, given appropriate emphasis on evaluating and correcting the issues.
5
Objective 2: Evaluate the project office’s issues management processes for: a) categorizing findings based on risk and priority; b) ensuring that relevant management findings are effectively communicated to the contractors; and c) ensuring that problems are evaluated and corrected on a timely basis. Both PPPO-M-414.1-5E, Quality Assurance Program Plan, and PPPO-M-414.1-1, Corrective Action Program, use the all-inclusive term “condition adverse to quality” (CAQ) for problems, failures, malfunctions, deficiencies, defective items, and nonconformances. The corrective action program (CAP) document additionally states that CAQs may be reported as findings, observations, significant CAQs, problem reports, judgments of need, or nonconformances. A significant CAQ is defined as a quality that, if left uncorrected, could have a serious effect on safety, the environment, or operability. No further process is identified to categorize the findings based on risk and priority. (See Opportunity for Improvement OFI- 3.) For issues categorized as high-significance findings (significant CAQs), the CAP procedure and the surveillance and assessments procedure require that an extent-of-condition and root cause analysis be performed, and that a cause code be assigned for all identified causes. They further require that corrective actions (CAs) should address all aspects of each identified causal factor and should provide both remedial actions to address the finding and preventive actions to prevent recurrence. A similar process is followed for CAQs, using a graded approach for the level of causal analysis. Section 5.7 of the procedure addresses verification of CAs, including effectiveness verifications. Although there is some evidence that effectiveness reviews are included in follow-up assessments, interviews with the staff indicated that these reviews primarily verify that the CAs have been completed, not that they are effective. (See Opportunity for Improvement OFI-4.) Tracking of CAs resulting from management assessments is addressed in the CAP procedure, which states that QA personnel enter the CAs into the QA tracking system. Tracking of CAs resulting from surveillances is not specifically addressed in the procedure on the assessment and surveillance process; however, the DUF6 Project oversight plan for operations and maintenance states that verification of CAs will be tracked as part of the DUF6 Project Letter Transmittal Tracking. It further states that tracking and verification of issues are the responsibility of the FIR author. Interviews with PPPO personnel indicate that some activities are tracked in the management tracking system, but most of the staff do not use the system. No overall issues management tracking system is used at PPPO. (See Opportunity for Improvement OFI-5.) The oversight process, as discussed in Objective 1, has identified relevant management findings. The issues identified as the result of management assessments are formally transmitted to the contractor, and the CAs are tracked in the QA tracking system. Surveillances and operational awareness activities are documented on FIRs, which are provided to the contractor electronically. The IPT submits a monthly letter to the contractor containing all FIRs, with an action response for “marginal” or “unsatisfactory” FIRs due within 15 days of receipt of the letter. The monthly letters are transmitted to the contractor by the FPD, who is a contracting officer’s representative. Independent Oversight reviewed a sample of the transmittal letters for assessment reports and FIRs and found them to effectively communicate the issues to the contractor. However, in the absence of a well-defined process for categorizing findings based on risk and priority, the significance of the issue is not fully communicated to the contractor. One example of the FR’s oversight activities involved observation of the calibration of process offgas inline oxygen sensors. The contractor identified the need to verify the seal leg water level once per shift and issued Standing Order Number SO-OPS-12-005. The standing order required the level of the seal leg water to be verified as greater than or equal to 48 centimeters. However, the units should have been in inches; i.e., the level should be verified to be greater than or equal to 48 inches. The FR noted this
6
discrepancy to the operations manager, who revised the standing order. While these actions addressed the immediate issue, this issue was not formally transmitted to the contractor. Further, it is indicative of a broader issue: The company that provided the process conversion equipment installed metric units on the equipment, but the balance-of-plant contractor used the English system of measurement. (See Opportunity for Improvement OFI-6.) Regarding the evaluation and correction of problems on a timely basis, the PPPO DUF6 IPT has identified and formally transmitted to the contractor concerns about the CONOPS program and the condition reporting program. The contractor’s initial response to these concerns was less than adequate, resulting in additional follow-up and rework. The first direction to the contractor to develop a CONOPS improvement plan was transmitted on September 26, 2011, with a due date within 15 days. BWCS replied on October 26, 2011, with a CONOPS improvement plan that lacked specificity and did not result in improved performance, as demonstrated by the number of findings identified during the focused CONOPS inspections conducted by PPPO in January and February 2012. A follow-up letter was submitted to BWCS on February 29, 2012, requesting that the existing CONOPS improvement plan be modified to address the additional issues, no later than 15 days from receipt of the letter. On March 20, 2012, BWCS provided a status report. Similarly, FIRs were formally transmitted to the contractor on October 25, 2011, identifying concerns about the condition reporting system. On December 21, 2011, the contractor committed to implementing a condition reporting process improvement plan. On January 19, 2012, the condition reporting process improvement plan was formally transmitted to PPPO; it consisted of a project schedule chart, with no supporting documentation. PPPO continued to identify concerns about the condition reporting system in a February 9, 2012, letter to the contractor, with a 15 day response request. The contractor provided a response to this letter on March 16, 2012, but still had not presented the condition reporting improvement plan to the project office. This sequence of events demonstrates a concern about the contractor’s ability to evaluate and correct problems on a timely basis. (See Opportunity for Improvement OFI-2.)
5.0 CONCLUSIONS The combination of PPPO and DUF6 Project procedures and documents that govern the site’s oversight process, and the process for scheduling and conducting activities, provide an acceptable basis for establishing an oversight program at the DUF6 conversion plants. Effective oversight is provided by the FRs, technical support contractors, NSOL, SSO, and the IPT. Concerns are appropriately identified through the assessment and operational awareness activities, and in most cases they are formally transmitted to the contractor for resolution. The management systems in place for oversight are appropriately designed to result in improved contractor performance, given appropriate emphasis on evaluating and correcting the issues. However, one finding was identified related to the need to periodically review and update management documents; several of them referenced outdated orders and standards. In addition, the assessment planning process does not sufficiently consider the effectiveness of contractor assurance systems, the hazards at the site/activity, and the degree of risk (e.g., giving additional emphasis to potentially highconsequence activities). The PPPO/DUF6 processes for issues management provide for categorization of CAQs as findings, observations, significant CAQs, problem reports, judgments of need, or nonconformances. However, there is no further categorization of findings based on risk and priority. The processes for communicating management issues to the contractor are effective; the results of assessment reports are formally transmitted to the contractor, and the FIRs are formally transmitted to the contractor from the FPD on a monthly basis. PPPO has recognized a concern that the contractor’s initial response to issues concerning
7
the CONOPS program and the condition reporting program did not contain adequate detail and have not resulted in improvements to the programs, The FPD has communicated to the contractor the need to improve in these areas. Opportunities to improve the issues management process include the assessment planning process, the issue categorization process, the effectiveness review process, and the management tracking systems.
6.0 FINDINGS AND OPPORTUNITIES FOR IMPROVEMENT During the review, Independent Oversight identified several issues, which were characterized in accordance with the PPPO procedure for assessment and surveillances and are annotated by type (i.e., a finding or observation). The PPPO issues management process defines a CAQ as an “an all-inclusive term used in reference to any of the following: problems, failures, malfunctions, deficiencies, defective item, nonconformance. These may be reported as findings, observations, significant CAQ, or other similar terms.” Observations closely approximate opportunities for improvement, which according to Independent Oversight protocols “are suggestions offered by the Independent Oversight appraisal team that may assist line management in identifying options and potential solutions to various issues identified during the conduct of the appraisal.” The finding and opportunities for improvement are summarized below and are provided to PPPO/DUF6 for evaluation and follow-up in accordance with PPPO procedures and processes. Independent Oversight identified one finding and six opportunities for improvement (observations). Findings F-1: Many key PPPO process procedures include outdated references to DOE orders and standards and the Functions, Responsibilities and Authorities list has not been revised as required in five years. Opportunities for Improvement OFI-1: Strengthen the process for developing the annual integrated assessment schedule to ensure appropriate focus on higher-risk areas. OFI-2: Investigate the use of additional measures (e.g., award fee, performance metrics, specific direction) to reinforce the expectation that the evaluation of CONOPS problems be performed in a high quality manner resulting in improvement, and that problems be corrected expeditiously. OFI-3: Develop a well documented issues categorization process that prioritizes findings based on risk and priority and improves communications to the contractor of the significance of the issues. OFI-4: Establish an effectiveness review process that includes effectiveness reviews in the integrated assessment schedule and incorporates training of staff. OFI-5: Investigate the use of an issues management tracking system, such as Orion or Pegasus. OFI-6: Direct BWCS to conduct a human factors evaluation of the use of both English and metric systems.
8
7.0 FOLLOW-UP ITEMS As part of its regular oversight activities, Independent Oversight will follow the closure of the finding identified above and monitor the execution of the CONOPS program at the DUF6 conversion plants.
9
Appendix A Supplemental Information Dates of Review Onsite Review: March 12-16, 2012
Office of Health, Safety and Security Management Glenn S. Podonsky, Chief Health, Safety and Security Officer William A. Eckroade, Principal Deputy Chief for Mission Support Operations John S. Boulden III, Director, Office of Enforcement and Oversight Thomas R. Staker, Deputy Director for Oversight William E. Miller, Deputy Director, Office of Safety and Emergency Management Evaluations Quality Review Board William Eckroade John Boulden Thomas Staker William Miller Michael Kilpatrick George Armstrong Robert Nelson Independent Oversight Site Lead for Portsmouth/Paducah Project Office Joseph Drago Independent Oversight Reviewers Joseph Drago – Lead Terry Olberding
A-1
Appendix B Documents Reviewed, Interviews, and Observations Documents Reviewed • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • BWCS Paducah Facility Manager and Shift Supervisor Narrative Log BWCS Standing Order SO-OPS-12-005, Seal Leg Water Levels, March 13, 2012 BWCS Standing Order SO-OPS-12-005, Seal Leg Water Levels, March 13, 2012. Rev 1 BWCS-C-MNT-0614, Functional Testing of Conversion Unit High Pressure, Condenser High Temperature, Off Gas High Pressure, and Oxygen Indicators, R1, 3/12/12 C-201109-008, Field Inspection Report, Paducah Post-Start Readiness Review Findings, 9/8/11 C-201109-10, Paducah Condition Report Closure Review – second quarter, 9/14/11 C-201109-11, Paducah Field Inspection Report, Condition Report Closure Review – third quarter, 9/12/11 Daily Communication and Teamwork Report, 3/13/12 Daily Communication and Teamwork Report, 3/14/12 DE-AC30-11CC40015, Contract exhibit 3 award fee-base section rating criteria worksheets DOE O 226.1B, Implementation of Department of Energy Oversight Policy, 4/25/11 DOE O 422.1, Conduct of Operations, 6/29/10 DOE O 5480.19, Conduct of Operation Requirements for DOE Facilities, 10/23/2001 DOE O 450.2, Integrated Safety Management, 4/25/11 DUF6 Conversion Project Assessment Schedule – Annual Look-A-Head, 4/15/11 DUF6 Conversion Project Assessment Schedule – Semi-Annual Look-A-Head, 12/30/11 DUF6 Conversion Project Assessments/Audits/Surveillance Schedule October 2009September 2010, 10/4/10 Email from Gregory Bazzell to Daniel Whitley, subject comments on two occurrence repots, 12/15/11 Email from Gregory Bazzell to Daniel Whitley, subject comments on occurrence report, 2/23/12 Email from Jim Kenney to Charles Armitage, subj: Modifications to CONOPS plan, 10/20/11 Email from Jim Kenney to Peter Burban, subj: CONOPS Surveillance plan for next 8 weeks, 1/4/12 Email from Michelle Reichert to Haug, et al, subj: Rollup of Safety Culture Assessment Results, 2/8/12 Email Robinson to Bazzell, Rollup of Safety Culture Assessment Results, March 15, 2012. Facility Specific Qualification Card, Portsmouth Conversion Facilities, 3/23/10 Facility Specific Qualification Card, Paducah Conversion Facilities, 8/27/10 FR/SSO Weekly Meeting Summary, 2/22/2012 FR/SSO Weekly Meeting Summary, 2/29/2012 Fy2012 PPPO Assessment Schedule Handout, DUF6 Monthly Project Review, 3/14/12 Handout, DUF6 Project Performance Metrics, FY 2012 through 2/12 Letter from Dials to Zimmerman, subj: Combined Integrated Safety Management System (ISMS) and Quality Assurance Declaration – FY 2011, 11/30/11
B-1
• • • • • • • • • • • • • • • • • • • • • • • • • • • • •
Letter from Dials to Zimmerman, subj: Conduct of Operations Improvement Plan, 10/26/11 Letter from Dials to Zimmerman, subj: DUF6 Conduct of Operations Improvement Plan, 3/20/2012 Letter from Dials to Zimmerman, subj: Improved Use of the Condition Report (CR) System for Operations and Maintenance Issues, 3/16/2012 Letter from Dials to Zimmerman, subj: Response to “Forwarding of Unsatisfactory December 2011 Field Inspection Report” letter, 1/19/12 Letter from Dials to Zimmerman, subj: Response to “Insufficient Response to Marginal and Unsatisfactory September 2011Field Inspection Report Letter, dated 12/21/11 Letter from Reichert to Zimmerman, subj: Safety Culture Assessment for the Piketon and Paducah DUF6 Facilities, 12/2/11 PADU-10-IA-056, Management Assessment of PPPO Facility Representative Program, 3/10 Paducah FR Weekly Report, 2/20/12 – 2/26/12 Paducah FR Weekly Report, 2/27/12 – 3/2/12 Paducah FR Weekly Report, 3/5/12 – 3/11/12 Paducah Report for Week Ending 2/24/12 Paducah Report for Week Ending 3/2/12 Paducah Report for Week Ending 3/9/12 PMP-5-08, DUF6 Conversion Project Oversight Plan – Operations and Maintenance, R3, 3/5/12 PMP-5-08, DUF6 Conversion Project Oversight Plan – Transition to Operations, R2, 8/11/10 Portsmouth Weekly Report 2/20/12 – 2/26/12 Portsmouth Weekly Report 2/27/12 – 3/4/12 Portsmouth Weekly Report 3/5/12 – 3/11/12 PPPO Functions, Responsibilities and Authorities, 1/25/2007 PPPO-01-1130-11, letter from Zimmerman to Dials, subj: Forwarding of Unsatisfactory September 2011 Piketon Field Inspection Report, 9/26/11 PPPO-01-1133-11, letter from Zimmerman to Dials, subj: Forwarding of September 2011 Field Inspection Reports and Surveillance Reports, 9/29/11 PPPO-01-1136-11, letter from Zimmerman to Dials, subj: Nuclear Safety Culture Management Assessment Report #PADU-11-MA-100329, R1, 9/29/11 PPPO-01-1139, letter from Zimmerman to Dials, subj: Forwarding of Unsatisfactory September 2011 Field Inspection Reports, 10/6/11 PPPO-01-1149-12, letter from Zimmerman to Dials, subj: Insufficient Response to Marginal and Unsatisfactory September 2011 Field Inspection Report Letter, 10/25/11 PPPO-01-1161-12, Letter from Zimmerman to Dials; subj: Qualification and Training Program Management Assessment Report PPPO-11-MA-100331, R2, 11/16/11 PPPO-01-1169-12, Letter from Zimmerman to Dials, subj: Forwarding of October 2011 Field Inspection Report, 11/21/11 PPPO-01-1171-12, letter from Zimmerman to Dials, subj: Forwarding of November 2011 Field Inspection Reports, 12/14/11 PPPO-01-1178-12, letter from Zimmerman to Dials, subj: Forwarding of Unsatisfactory December 2011 Field Inspection Report, 12/20/11 PPPO-01-1179-12, Letter from Zimmerman to Dials, 12/22/11; subj: Authorized Release of Aqueous Hydrofluoric Acid Program 2nd Quarterly Assessment Report #PADU-11MA-100344, R1
B-2
• • • • • • • • • • • • • • • • • • • • •
PPPO-01-1187-12, Letter from Zimmerman to Dials; subj: Integrated Project Team Oversight Assessment Schedule and Near-Term Planned Assessments, 1/11/12 PPPO-01-1189-12, letter from Zimmerman to Dials, subj: Forwarding of Marginal December 2011 Field Inspection Reports, 1/11/12 PPPO-01-1388916-12, Letter from Zimmerman to Dials, subj: Forwarding of January 2012 Field Inspection Reports, 2/8/12 PPPO-01-1390541-12, letter from Zimmerman to Dials, subj: Improved Use of the Condition Report System for Operations and Maintenance Issues, 2/9/12 PPPO-01-1390541-12, letter from Zimmerman to Dials, subj: Improved Use of the Condition Report System for Operations and Maintenance Issues, 2/9/12 PPPO-01-1413751-12, Letter from Zimmerman to Dials, subj: Forwarding of Informational January and February 2012 Field Inspection Reports, 2/29/12 PPPO-01-981250-10B, memo from Murphie to file, subj
UF6 Facility Representative Qualifications, 8/12/10 PPPO-1063, Facility Representative Program Plan, R1, 3/10 PPPO-M-226.1-2, Oversight Program Plan, R1, 3/10 PPPO-M-413.1-1, Management Plan, R3, 3/10 PPPO-M-414.1-1, Corrective Action Program, R1, 3/2010 PPPO-M-414.1-2, Assessment and Surveillance Process, R1 3/2010 PPPO-M-414.1-5E, Quality Assurance Program Plan, R5, 2/2010 PPPO-M-420.1-3, Safety Systems Oversight Program Plan, R0, 2009 Report, DOE Assessment Drivers Safety System Oversight Qualification Card, 2/25/10 Support Contractor Qualification Card, 3/7/11 Various ORPS reports X-201009-24, Field Inspection Report, Portsmouth, Self-Assessment of FIRs, 9/10-20/10 X-201109-011, Portsmouth, Field Inspection Report, Portsmouth Post-Start Readiness Review Findings, 9/2/11 X-201109-12, Piketon Field Inspection Report, Condition Reporting, 9/12/11Interviews Nuclear Safety Oversight Lead DOE Facility Representative for DUF6 Paducah DOE Facility Representative for DUF6 Portsmouth BWCS Paducah Plant Manager BWCS Portsmouth Plant Manager Federal Project Director Deputy Federal Project Director for Paducah Deputy Federal Project Director for Portsmouth DOE Safety System Oversight Engineer PRC Support Contractor Lead PRC Support Contractor QA PRC Support Contractors Paducah PRC Support Contractors Portsmouth
B-3
Observations DUF6 Paducah plant tour Monthly Project Status meeting Daily Communication and Teamwork meeting Shadow of FR walkthrough
B-4
doc_785864016.pdf