FINAL SUPPORTING STATEMENT FOR
COLLECTION OF OPERATOR SIMULATOR TRAINING DATA (3150-0234)
EXTENSION
Description of the Information Collection
This information collection request is to the holders of, or applicants for, a power reactor operating license under Title 10 of the Code of Federal Regulations (10 CFR) Part 50, “Domestic Licensing of Production and Utilization Facilities,” except those that have certified that they have permanently ceased operations and have permanently removed all fuel from the reactor vessel, and the holders of, or applicants for, a power reactor combined license under 10 CFR Part 52, “Licenses, Certifications, and Approvals for Nuclear Power Plants.”
This information collection is for the specified licensees to use the NRC-developed Scenario Authoring, Characterization and Debriefing Application (SACADA) software for their operator simulator training. The SACADA system was developed to collect licensed operator simulator training data to inform human reliability analysis (HRA) and to facilitate operator simulator training. The SACADA software can be used to author the simulation scenarios, facilitate the post simulation debriefing on crew performance, guide performance analysis, and generate various types of reports. The information entered into the SACADA database can be used to improve simulator training effectiveness and HRA. The South Texas Project Nuclear Operating Company (STPNOC) has used the software for its operator simulator training since 2012 and highly regards the software. The NRC welcomes more licensees to partner with the NRC to use the software. The licensees’ participation in the information collection is voluntary. In the partnership, the NRC provides the SACADA software license, training, and technical support to the participating licensees, and the participating licensees grant NRC access to analyze the data to improve the NRC’s HRA techniques. An agreement will be developed to specify the details.
To participate in the information collection, the licensee will notify the NRC contact that it is interested in evaluating the software. Then the NRC will provide additional information including an onsite briefing. If the licensee thinks the SACADA software could be beneficial, the NRC will provide a training session, the software license, and technical support for the licensee to pilot the use of the software in its simulator training. After the pilot study, the licensee will decide whether or not to partner with the NRC on the information collection. Either party can terminate the agreement at any time.
JUSTIFICATION
Need For and Practical Utility of the Collection of Information
The HRA methods currently used by the NRC and industry do not have a strong data basis. Expert judgment played a dominant role in developing the methods. This caused problems such as large uncertainties in the estimation results due to a lack of data supporting the method development. The data collected in SACADA would
improve HRA methods by strengthening their data basis and reducing uncertainty to support NRC’s risk-informed decision making.
Agency Use of Information
The NRC will use the collected information from all participating licensees to generate human performance indications with statistical bases to improve the NRC’s HRA techniques. The human performance indications are expected to be generic (i.e., derived from all participating licensees’ data) instead of plant-specific. The NRC will not issue any enforcement action (finding or violation) to the participating licensees for non-willful violations identified as a result of weakness discovered in the licensees’ data. The use of data will be explicitly specified in the agreement for the partnership.
Reduction of Burden Through Information Technology
There are no legal obstacles to reducing the burden associated with this information
collection. In this information collection, all information is filed electronically through
emails (with the NRC contact) and the electronic information transmitting function is
provided in the SACADA software.
Effort to Identify Duplication and Use Similar Information
No sources of similar information are available. There is no duplication of requirements.
Effort to Reduce Small Business Burden
Not applicable.
Consequences to Federal Program or Policy Activities if the Collection Is Not Conducted or Is Conducted Less Frequently
The information collection is to use the NRC’s SACADA software to collect the operator performance information in simulator training. The information collection enables the NRC to analyze the operator simulator performance to improve NRC’s HRA methods to support the NRC’s risk-informed regulatory decision-making processes such as the Significance Determination Process of the Reactor Oversight Process. The collected information will improve the NRC’s HRA methods for more realistic and reliable results for regulatory decisions.
The collected information will be used to develop human performance indications based on all participating licensees’ data (i.e., generic indications instead of plant- specific indications). Not conducting or conducting the information collection with less frequency would reduce the amount of human performance information available and, thereby, limit the improvements to the realism of the NRC’s HRA methods.
Circumstances Which Justify Variation from OMB Guidelines
The following are the circumstances and justifications apply to the information collection:
Information is collected more often than quarterly: The licensees generally conduct five to six training cycles each year for simulator training. For a licensee participating in the information collection, the NRC expects the licensees to make the data available to the NRC at the end of each training cycle which is about once every two months. The SACADA software enables the licensee to share the information with the NRC simply and electronically.
Consultations Outside the NRC
The opportunity for public comment on the information collection requirements for this clearance package has been published in the Federal Register.
Opportunity for public comment on the information collection requirements for this clearance package was published in the Federal Register on September 11, 2020 (85 FR 56277). NRC received four comments from Arizona Public Service (APS).
Comment 1:
Is the proposed collection of information necessary for the NRC to properly perform its functions? Does the information have practical utility?
The exact data to be collected is not known. In the past, the NRC has been able to perform its functions without the need of specific simulator data, other than when evaluating human reliability analysis (HRA). Over the past four years, Arizona Public Service (APS) has been successful in gathering such information upon demand. Further, from the inception of the Reactor Oversight Process, the proposed collection did not exist and thus the NRC has been able to effectively perform its regulatory function to date without it.
Outside of HRA, the practical utility of such information is not clearly described, nor are limitations of use. The docket information describes the data to be collected as:
“…licensed operator simulator training data to inform human reliability analysis (HRA) and to facilitate operator simulator training. The SACADA software can be used to author the simulation scenarios, facilitate the post simulation debriefing on crew performance, guide performance analysis, and generate various types of reports.”
In the above statement, the information being collected is not clearly described, and the description is combined with a statement of the usefulness of the software, and not its capabilities, limitations, and specific data points collected.
Attempting to answer this question by inference would provide data that is not pertinent to the NRC’s regulatory function, for example, specific internal decision making in guidance of performance analysis and specific reports generated by the utility.
It is difficult to answer the questions based upon the information provided. APS would have to see the software and the information it provides. Providing this information could improve how the NRC assesses risk as operators mitigate events, however, the NRC does not need this information to perform their regulatory functions.
NRC Response:
The information collected by “Collection of Operator Simulator Training Data – SACADA” directly supports the development of HRA methods that are used in performing functions that regulate licensees, such as licensing, oversight, ASP, and SDP. Data for HRA is critical for improving realism of the analyses and for analyzing areas not currently represented by existing HRA methods (i.e., FLEX).
Comment 2:
Is the burden estimate accurate?
No. The time estimates do not accurately account for the time it takes to formally respond to the Regulatory Issue Summary (RIS) after each training cycle and the time it takes to formally provide information to the NRC. APS takes time to ensure all of the documents are accurate and properly reviewed (3 people at a minimum). The process of getting a letter signed and submitted after each cycle would be 4-8 man hours for each RIS following a licensed operator continuing training (LOCT) cycle. APS believes this information intended for research will impose an administrative burden.
It is unclear if the information from this request would be used for other NRC inspections, specifically Inspection Procedure 71111, Attachment 11 (IP 71111.11), Licensed Operator Requalification Program and Licensed Operator Performance. In this inspection, simulator performance under a variety of instances is evaluated against the reference unit to ensure the requirements of ANSI/ANS-3.5, Nuclear Power Plant Simulators for Use in Operator Training and Examination, are met. It is believed that the information collected by this program could be used for these inspections which creates demand for additional reviews of the information collected from the simulator and submitted to the NRC. If this request is used for IP 71111.11 inspection preparations, this could impose additional administrative burden.
NRC Response:
NRC is not putting out a RIS, and the responses would only be associated with parties that wish to participate in the program. After that, we can say how much time it typically takes to respond submit data to the NRC in the program. Also, we will emphasize that this information is not used for inspection programs.
Comment 3:
Is there a way to enhance the quality, utility, and clarity of the information to be collected?
APS does not use and has not had any training on the SACADA software. Without any training or having actually used the SACADA utility, APS does not know if there is a way to enhance the quality, utility, and clarity of the information to be collected. The information contained in the Federal Register (FR) Notice, NRC-2020-0064, does not provide enough detail to answer this question. Clarification on what specific information is being collected and how it’s used is needed to answer this question.
NRC Response:
SACADA provides detailed guidance to lead people to understand how to use it. If it is needed, NRC also can provide training to learn how to use SACADA to collect information.
Comment 4:
How can the burden of the information collection be minimized, including the use of automated collection techniques or other forms of information technology?
Ultimately, making this process completely automated could minimize the burden of the information being collected. The burden estimate is assumed to be 128 hours each year. Burden estimates are only for the NRC to utility contact for billing information. This estimate does not take into account PVNGS internal burden, which would be considerable, especially with the additional management and leader reviews that would go into reviewing data inputs and outputs of SACADA.
Additional clarification is needed to determine if the information requested can and will be used during the implementation of IP 71111.11. Without understanding how the simulator data is used, all simulator data would require evaluations at a depth not normally performed for LOCT cycle. In depth analysis of simulator performance is typically performed during simulator testing in scenarios specifically stated in ANSI/ANS-3.5 and when needed based upon operator feedback.
NRC Response:
The agreements are voluntary initiatives, there is no billing of services between the NRC and the industry. Further, the data inputs and outputs are equivalent to what is already spent in the licensees training programs, so there is negligible added cost by using the SACADA database. The amount of time spent to transmit the data to the NRC is minimal. This is not used in inspection programs.
NRC also contacted nine licensees by email and there were zero comments received.
Payment or Gift to Respondents
Not Applicable.
Confidentiality of Information
Confidential and proprietary information is protected in accordance with the NRC regulations in 10 CFR 9.17(a) and 10 CFR 2.390(b).
Justification for Sensitive Questions
Not Applicable.
Estimated Burden and Burden Hour Cost
One-time burdens
The NRC staff estimates that 58 will review the NRC REGULATORY ISSUE SUMMARY 2017-01, REVISION 1, “HUMAN RELIABILITY AND HUMAN PERFORMANCE DATABASE”, (OMB Control No.: 3150-0234), (ML17128A343) soliciting participants to use the SACADA software. This burden is estimated to take 1 hour for each licensee (58 hours).
The NRC staff estimates that 5 licensees will opt to participate. Their response to the RIS will take approximately 30 minutes (1.5 hrs).
The total burden is estimated to be 20 hours (58 hours + 1.5 hours =
59.5 hours, annualized to 20 hours).
Ongoing annual burdens
The 5 participating licensees will incur the following burdens annually:
Developing, acquiring, installing, and utilizing technology and systems for the purpose of collecting, validating, and verifying information = 20 hours (4 hrs x 5 licensees)
Developing, acquiring, installing, and utilizing technology and systems for the purpose of processing and maintaining information = 10 hours (2 hours x 5 licensees)
Developing, acquiring, installing, and utilizing technology and systems for the purpose of disclosing and providing information = 2.5 hrs (0.5 hrs x 5 licenses)
Training personnel to be able to respond to a collection of information = 80 hours (16 hours x 5 licensees)
Transmitting, or otherwise disclosing the information = 15 hours (0.5 hours x 5 licensees x 6 training cycles annually)
The total ongoing reporting burden associated with SACADA is 128 hours (20 hrs + 10 hrs + 2.5 hrs + 80 hrs + 15 hrs).
TOTAL SACADA burden
The total annual burden associated with this information collection during the three-year clearance period is 148 hours (20 one-time reporting burden + 128 annual reporting burden.)
The total number of responses is 32 annual responses ( 5 one-time responses [annualized to 2] + 30 annual responses [5 licensees x 6 annual responses])
Using the fee rate of $278 per hour for 148 hours, the estimated burden cost is $ 41,144 (148 hours x $278/hr).
The $278 hourly rate used in the burden estimates is based on the Nuclear Regulatory Commission’s fee for hourly rates as noted in 10 CFR 170.20 “Average cost per professional staff-hour.” For more information on the basis of this rate, see the Revision of Fee Schedules; Fee Recovery for Fiscal Year 2019 (84 FR 22331, May 17, 2019).
Estimate of Other Additional Costs
There are no additional costs.
Estimated Annualized Cost to the Federal Government
The NRC solicited the services of a contractor to maintainthe SACADA database and provide technical support with the average annual cost of $254,000.
The staff has developed estimates of annualized costs to the Federal Government related to the conduct of this collection of information. These estimates are based on staff experience and subject matter expertise and include the burden needed to review, analyze, and process the collected information and any relevant operational expenses.
Reasons for Change in Burden or Cost
The estimated annual burden for the information collection has decreased from 411 hours and 95 responses to 148 hours and 32 responses, a decrease of 263 hours and 63 responses. The change in annual burden is the result of staff’s expectations that:
1. Fewer licensees will annually review the Regulatory Information Summary (RIS). The RIS initiating the information collection was issued in 2018. Many licensees reviewed the RIS at that time. The NRC staff estimates that 58 licensees and applicants will review the RIS during the current clearance period, compared to 76 during the previous clearance period.
2. The expected number of licensees annually participating in the data collection changed from 15 to 5 including new and current participants.
3. The fee rate changed from $267 to $278 per hour
Publication for Statistical Use
Data analysis results will be published electronically and made available in ADAMS.
Reason for Not Displaying the Expiration Date
The expiration date will be displayed.
Exceptions to the Certification Statement
None.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | FINAL SUPPORTING STATEMENT |
| Author | keb1 |
| File Modified | 0000-00-00 |
| File Created | 2021-02-03 |