Inspection Manual Chapter 0608, Performance Indicator Program

NRC INSPECTION MANUAL
INSPECTION MANUAL CHAPTER 0608

PERFORMANCE INDICATOR PROGRAM

IRAB

TABLE OF CONTENTS
0608-01

PURPOSE .......................................................................................................... 1

0608-02

OBJECTIVES ...................................................................................................... 1

608-03

APPLICABILITY ................................................................................................... 1

0608-04

DEFINITIONS ...................................................................................................... 1

0608-05
05.01
05.02
05.03
05.04

RESPONSIBILITIES AND AUTHORITIES ........................................................... 2
Director, Office of Nuclear Reactor Regulation (NRR) .......................................... 2
Director, Division of Inspection and Regional Support (DIRS) .............................. 2
Chief, Reactor Assessment and Human Factors Branch (IRAB) .......................... 2
Regional Administrator (RA) ................................................................................ 2

0608-06

REQUIREMENTS ................................................................................................ 2

0608.07
07.01
07.02

GUIDANCE .......................................................................................................... 3
ROP Framework Background .............................................................................. 3
PIs ....................................................................................................................... 3

0608-08
08.01
08.02
08.03

PI DATA SUBMISSION AND PROCESSING ...................................................... 4
Reporting PI Data………………………..................................................... ……….4
Posting PI Data to NRC Web Sites……… .......................................... …………...5
PI Data Submission for Plants in Extended Shutdown ....................... …………...5

0608-09
09.01
09.02
09.03

PI VERIFICATION .............................................................................................. 5
Verification……………………… ............................................................... ……….5
Discrepant or Unreported PIs ……… ................................................. …………...5
Extended Shutdowns……… .............................................................. …………...6

0608-10
10.01
10.02
10.03
10.04

FEEDBACK AND DIFFERENCES IN INTERPRETATION .................................. 6
Differences in Interpretation of NEI 99-02………………………................. ……….6
FAQ Process ……… .......................................................................... …………...7
General Feedback about the PI Program……… ................................ …………...8
Closure ……………………………………… ................................... ……………...13

0608-11

REFERENCES ..................................................................................................14

Attachment 1: Flowchart for Addressing Questions and Feedback Related to ROP Performance
Indicator
Attachment 2: Revision History for IMC 0608

Issue Date: 02/05/9

i

0608

0608-01

PURPOSE

This Inspection Manual Chapter (IMC) provides guidance on the implementation of the Reactor
Oversight Process (ROP) Performance Indicator (PI) Program.

0608-02

OBJECTIVES

02.01 To provide policy and guidance regarding implementation of the ROP PI Program,
including the submission and verification of PI data and the posting of PI data and frequently
asked questions (FAQs) on NRC Web sites.
02.02 To establish a formal process for responding to questions related to the interpretation of
PI reporting guidance.
02.03 To establish a formal process for developing and implementing changes to the PI
Program, including creating new PIs and changing existing PIs.

0608-03

APPLICABILITY

This IMC applies to all operating commercial nuclear power reactors.

0608-04

DEFINITIONS

04.01 Extended Shutdown. For the purposes of the PI Program, an extended shutdown is a
condition in which a nuclear power reactor has been subcritical for at least six months.
04.02 Frequently Asked Question. An ROP PI FAQ is a question or a requested change from
an external stakeholder regarding the PI Program or its implementation. An FAQ is submitted to
the ROP Working Group (WG) in accordance with NEI 99-02.
04.03 NEI 99-02. The current revision of Nuclear Energy Institute (NEI) 99-02, “Regulatory
Assessment Performance Indicator Guideline,” is a document published by NEI that contains
guidance for calculating and reporting PI data. NEI 99-02 is jointly produced by the U.S.
Nuclear Regulatory Commission (NRC) and NEI.
04.04 Performance Indicators. PIs are objective data regarding licensee performance in the
ROP cornerstones of safety and security.
04.05 PI Discrepancy. A PI discrepancy is a difference between what was supposed to be
reported in accordance with the current NRC-accepted version of NEI 99-02 and what was
reported by a licensee in its PI data submittal.
04.06 ROP Feedback Form. An ROP feedback form (FBF) is Exhibit 1 of IMC 0801,
“Inspection Program Feedback Process.”

Issue Date: 02/05/19

1

0608

04.07 ROP Working Group. The ROP WG is an assembly of NRC staff and commercial
nuclear power industry representatives who meet periodically in a public meeting to discuss
FAQs and other issues related to ROP programs.
04.08 White Paper. A white paper is a document created by any stakeholder that contains
proposed generic changes to NEI 99-02 or the PI Program and is presented to the ROP WG.

0608-05
05.01

RESPONSIBILITIES AND AUTHORITIES
Director, Office of Nuclear Reactor Regulation (NRR).

a.

Provides overall policy direction for the PI Program

b.

Directs the development, and implementation of policies, programs, and procedures for
the PI Program and oversight of program effectiveness and implementation

05.02

Director, Division of Inspection and Regional Support (DIRS).

a.

Manages PI Program development, and implementation within NRR and oversees
program implementation and effectiveness

b.

Makes the final decision on an FAQ resolution when the ROP WG cannot reach
alignment

05.03

Chief, Reactor Assessment and Human Factors Branch (IRAB).

a.

Develops policy, programs, and procedures for implementation of the PI Program

b.

Receives and posts PI data and FAQs on NRC Web sites

c.

Manages and implements the process for responding to questions related to
interpretation of PI reporting guidance and develops and implements changes to the PI
Program, including creating new PIs and making changes to existing PIs or thresholds

d.

Assesses PI Program effectiveness and implementation

05.04 Regional Administrator (RA). The RA manages regional implementation of the PI
Program in accordance with the requirements of this IMC, Management Directive (MD) 8.13,
“Reactor Oversight Process,” Inspection Procedure (IP) 71150, “Discrepant or Unreported
Performance Indicator Data,” and IP 71151, “Performance Indicator Verification.”

0608-06

REQUIREMENTS

There are no requirements in this document. This document is for guidance only.

Issue Date: 02/05/19

2

0608

0608-07

GUIDANCE

07.01 ROP Framework Background. The ROP is built upon a framework directly linked to the
NRC’s mission. That framework includes cornerstones of safety and security that focus on the
licensee’s ability to (1) limit the frequency of initiating events; (2) ensure the availability,
reliability, and capability of mitigating systems; (3) ensure the integrity of the fuel cladding,
reactor coolant system (RCS), and containment; (4) ensure the adequacy of the emergency
preparedness functions; (5) protect the public from exposure to radioactive material releases;
(6) protect nuclear plant workers from exposure to radiation; and (7) provide assurance that a
licensee’s security system and material control and accounting program can protect against the
design basis threat of radiological sabotage and the theft or loss of radiological materials. The
ROP cornerstones are more fully described in IMC 0308, “Reactor Oversight Process Basis
Document.”
Within each cornerstone, a broad sample of data on which to assess licensee performance in
risk-significant areas is gathered from PI data submitted by licensees and from the NRC’s riskinformed baseline inspections. The PIs are not intended to provide complete coverage of every
aspect of plant design and operation, but they are intended to be indicative of performance
within the related cornerstone.
Data submitted by each licensee are used to calculate PI values. These values are then
compared to objective thresholds to determine the performance band associated with those
values. The performance bands are color-coded. Plant data for a PI that falls within the “green”
band indicate licensee performance is within the nominal, expected range. The “white” band
indicates that performance is outside of the nominal, expected range and can be characterized
as of low to moderate safety significance, but performance remains acceptable. Performance in
the “yellow” band indicates a more significant decline in performance and can be characterized
as being of substantial significance. Performance is considered acceptable, but a reduction in
safety margin exists. Performance in the “red” band indicates a very significant decline in
performance. Changes can be characterized as being of high safety significance. Performance
may be acceptable with a significant reduction in safety margin or may be unacceptable.
07.02 PIs. IMC 0308, Attachment 1, “Technical Basis for Performance Indicators,” and
IMC 0308, Attachment 6, “Basis Document for Security Cornerstone of the Reactor Oversight
Process,” describe the PIs; their objectives, thresholds, and bases; and ROP cornerstone
attributes covered by the PIs. NEI 99-02 describes the PIs, how they are calculated, and how
and when to report PI data to the NRC. NRC Regulatory Issue Summary (RIS) 2000-08,
“Voluntary Submission of Performance Indicator Data,” Revision 1, informs stakeholders that
the NRC accepts NEI 99-02 for use in reporting PI data. The latest revision of NEI 99-02
accepted by the NRC for use in reporting PI data is posted on the NRC’s public Web site.
PIs are a means of obtaining information related to licensee performance in certain attributes of
each cornerstone. They provide indication of problems that, if uncorrected, may increase the
probability and/or the consequences of an off-normal event. Because not all aspects of licensee
performance can be monitored by PIs, safety and security significant areas not covered by PIs
are assessed using the ROP Inspection Program.
The ROP cornerstones and the current suite of PIs that monitor performance in some of the
cornerstones’ attributes are as follows.

Issue Date: 02/05/19

3

0608

1.

Initiating Events Cornerstone:
•
IE01: Unplanned Scrams per 7,000 Critical Hours
•
IE03: Unplanned Power Changes per 7,000 Critical Hours
•
IE04: Unplanned Scrams with Complications

2.

Mitigating Systems Cornerstone:
•
MS05: Safety System Functional Failures (SSFFs)
•
Mitigating System Performance Index (MSPI). The MSPI is calculated separately
for each of the following five systems for each reactor type:
•
MS06: Emergency AC Power Systems
•
MS07: High Pressure Injection Systems. For pressurized water reactors
(PWRs), the high pressure safety injection system is monitored. For boiling
water reactors (BWRs), the high pressure coolant injection system (e.g., high
pressure coolant injection, high pressure core spray, and/or feedwater coolant
injection) is monitored.
•
MS08: Heat Removal Systems. For PWRs, the auxiliary feedwater system is
monitored. For BWRs, the heat removal systems monitored can include the
reactor core isolation cooling and/or isolation condenser systems.
•
MS09: Residual Heat Removal Systems (or the equivalent function)
•
MS10: Cooling Water Support Systems (for the above systems)

3.

Barrier Integrity Cornerstone:
•
BI01: RCS Specific Activity
•
BI02: RCS Identified (or Total) Leakage

4.

Emergency Preparedness Cornerstone:
•
EP01: Drill/Exercise Performance
•
EP02: Emergency Response Organization Drill Participation
•
EP03: Alert and Notification System Reliability

5.

Occupational Radiation Safety Cornerstone:
•
OR01: Occupational Exposure Control Effectiveness

6.

Public Radiation Safety Cornerstone:
•
PR01: Radiological Effluent Technical Specifications/Offsite Dose Calculation
Manual Radiological Effluent Occurrences

7.
•

0608-08

Security Cornerstone:
PP01: Protected Area Security Equipment Performance Index

PI DATA SUBMISSION AND PROCESSING

08.01 Reporting PI Data. Reporting PI data to the NRC is a voluntary program in which
licensees of commercial nuclear power plants participate. The Office of Management and
Budget (OMB) clearance for PI reporting is OMB No. 3150-0195. The introductory section of
NEI 99-02 contains the guidance and due dates for reporting PI data.

Issue Date: 02/05/19

4

0608

08.02 Posting PI Data to NRC Web Sites. After the PI data are received by IRAB staff, they
are entered into the NRC’s Reactor Programs System database to calculate the indicator values
and generate NRC Web site files. The NRC will post the data, the indicator values, and
associated graphs on the NRC’s internal Web site. IRAB staff will notify the NRC regional
offices when the PIs are available on the NRC’s internal Web site so they can review the PIs
and identify any inconsistencies prior to public release. The NRC will then place the PIs on the
NRC’s external Web site to make them available to external stakeholders.
08.03 PI Data Submission for Plants in Extended Shutdowns. Commercial nuclear power
plants may be shut down for an extended period of time for a variety of reasons. For these
sites, the NRC may apply the process described in IMC 0350, “Oversight of Reactor Facilities in
a Shutdown Condition Due to Significant Performance and/or Operational concerns,” or the
NRC may apply the guidance in IMC 0351, “Implementation of The Reactor Oversight Process
at Reactor Facilities in an Extended Shutdown Condition for Reasons Other Than Significant
Performance Problems.” Because some PIs are heavily influenced by the operational status of
the reactor (e.g., the number of hours a reactor has been critical), these PIs may no longer
provide valid indications of performance during an extended shutdown. A licensee with a plant
in an extended shutdown should report PIs for that plant in accordance with the guidance
provided in the current revision of NEI 99-02. PIs that are invalid because the plant is in an
extended shutdown will be displayed as “not applicable” on NRC Web sites.

0608-09

PI VERIFICATION

09.01 Verification. PI data are voluntarily submitted by licensees to the NRC; however,
information provided to the NRC by a licensee must be complete and accurate in all material
respects. Because PI data are sources of information upon which NRC assessment and
oversight actions will be based, the failure to report PI data completely and accurately can
impede the regulatory process and therefore have traditional enforcement implications.
IP 71151 shall be conducted to review licensees’ PI data collection and reporting activities for
adherence to pertinent guidance. The NRC expects licensees to make reasonable, good faith
efforts to comply with the guidance in NEI 99-02. This includes taking appropriate and timely
action to identify and report performance issues captured by the indicators. It may be
necessary for inspectors to exercise some judgment on the adequacy of licensee actions to
make a reasonable, good faith effort to comply with the guidance.
09.02 Discrepant or Unreported PIs. Instances of PI discrepancies and unreported PIs should
be documented in accordance with IP 71151 and IMC 0612, “Power Reactor Inspection
Reports.” Enforcement action will be taken for incomplete or inaccurate PI reporting in
accordance with the NRC Enforcement Policy. If the NRC determines that PI discrepancies
exist that cause NRC staff to lose confidence in the licensee’s ability to collect and report PI
data accurately, the affected PI(s) will be classified as discrepant on the NRC Web sites, and
the staff will perform IP 71150. Factors to consider when deciding to perform IP 71150 include
whether the licensee is correcting the PI data errors, the effectiveness of those corrective
actions, the repetitiveness of the errors, and any trends in the quality of PI data reporting that
the NRC may be aware of. The decision to perform IP 71150 should be discussed (and could be
made) during the plant performance reviews described in IMC 0305, “Operating Reactor
Assessment Program.” IP 71150 provides guidance for collecting PI data and inspecting
cornerstone attributes to compensate for the discrepant or unreported PI data. Regional
management should coordinate activities in this area with NRR/DIRS. The selected inspections
will be performed in addition to the baseline inspection. Once the licensee has corrected the

Issue Date: 02/05/19

5

0608

root cause(s) of the discrepant or unreported data and the NRC has verified that the licensee
can collect and report PI data accurately, oversight of PI reporting in accordance with IP 71151
will resume.
09.03 Extended Shutdowns. When a plant has been in an extended shutdown, some PIs may
not provide a meaningful indication of plant performance in the cornerstone attributes they are
intended to monitor (i.e., the PIs become invalid or not applicable). In these situations, the
guidance provided in IP 71150 should be followed to obtain sufficient performance information
via the inspection program when possible until the plant has restarted and the PIs become valid.

0608-10

FEEDBACK AND DIFFERENCES IN INTERPRETATION

The NRC receives feedback and suggestions from various stakeholders about the PI program.
For example, an NRC inspector may submit an ROP FBF that recommends modifying a PI to
address possible unintended consequences; an ROP survey respondent may request the NRC
to change a PI threshold; or a lessons learned task force may suggest a new PI. In the spirit of
continuous improvement, NRC staff evaluates this feedback to determine if enhancements to
the PI Program are warranted.
Various stakeholders also submit questions regarding the interpretation of NEI 99-02. For
example, a licensee and an inspector may disagree over the interpretation of NEI 99-02 and
therefore seek clarification from the ROP WG. In these cases, NRC staff engages with the
internal and external stakeholders to interpret the guidance and determine if clarifications or
changes thereto are warranted.
This section describes a process to address such questions and feedback from internal and
external stakeholders. Attachment 2 of this IMC also summarizes this process using a
flowchart. This section describes actions taken for differences in interpretation of NEI 99-02
guidance, the FAQ process, general feedback about the PI Program, and close-out activities.
Some questions and issues (e.g., those involving other NRC regulatory documents or
programs) fall outside the scope of this process, even though the issue may affect PI data
values. For example, questions about a plant’s design or licensing basis, interpretation of
Technical Specifications, or reporting requirements should be directed to other NRC technical
leads or processes (e.g., the Task Interface Agreement process). An NRC’s Division of
Operating Reactor Licensing project manager can assist with referrals to other NRC technical
leads or processes.
10.01 Differences in Interpretation of NEI 99-02. If an NRC inspector and a licensee have
differing views about the interpretation of NEI 99-02 and approved FAQs that could involve a
potential PI discrepancy, the issue may need to be resolved at the ROP WG meetings. An NRC
inspector (or any NRC employee) should initiate the process by contacting the PI Program Lead
in NRR/DIRS. The inspector should be prepared to provide the PI Program Lead with a
description of the circumstances, the guidance in question, and necessary background
information.
NRC staff may also submit an ROP FBF to receive a more formal response (i.e., the FBF
resolution would have IRAB BC concurrence). ROP FBFs involving differing interpretations of
NEI 99-02 should indicate “IP 71151” in the IP/IMC section of the ROP FBF because it involves
an NEI document rather than an IMC or IP for which the ROP feedback

Issue Date: 02/05/19

6

0608

process was originally designed. The FBF should indicate the PI guidance in question,
document that the FBF is being submitted because of differing interpretations of NEI 99-02, and
list specific guidance references (e.g., NEI 99-02 page numbers and lines). Additional guidance
is provided in IP 71151 and in IMC 0801.
When the PI Program Lead receives an inquiry from an inspector seeking clarification of NEI 9902 guidance, the PI Program Lead, with assistance from other NRC technical leads if
necessary, and after consultation with the IRAB BC, will provide the inspector its initial
interpretation based on the information provided by the inspector. NRC staff should first seek
alignment to the extent possible. NRC staff should consider the clarity and the intent of the
guidance. The staff may refer to other NRC documents (e.g., IMC 0308, Attachment 1 and
ROP-related SECY papers) to inform its position.
After the PI Program Lead provides the NRC inspector with the initial NRC interpretation, the
inspector should then discuss the interpretation with the licensee. If the inspector and licensee
continue to have differing views, and the licensee does not submit an FAQ in a timely manner or
at all, then the inspector should follow the guidance in IP 71151 for a PI discrepancy (i.e.,
consider enforcement action). If the licensee submits an FAQ, the inspector should follow the
guidance in IP 71151 for inspection results and documentation, and the staff will follow the FAQ
process described in the next section of this IMC.
10.02 FAQ Process. NEI 99-02, Appendix E, establishes the FAQ process to resolve differing
interpretations of NEI 99-02, address unique situations for which NEI 99-02 is not clear, and
incorporate changes into NEI 99-02 after completion of the white paper process, which is
described in Section 09.03 of this IMC.
Industry stakeholders that are members of NEI submit FAQs directly to the ROP Task Force.
The ROP Task Force will follow their internal process before introduction of the FAQ at an ROP
WG meeting. Industry stakeholders that are not members of NEI submit FAQ directly to the
NRC’s PI lead, who will then coordinate the introduction of the FAQ at the next ROP WG
meeting.
The typical FAQ process is described as follows.
a.

Introduced. The industry introduces FAQs at an ROP WG meeting. If the FAQ involves
plant-specific security information, the ROP WG meeting will acknowledge the status of
the FAQ; however, a separate non-public meeting will be conducted to resolve the
FAQ. For FAQs involving differing interpretations of NEI 99-02, the NRC resident
inspector should plan to call into the ROP WG meeting to discuss her or his views on
the issue.
The NRC or industry may also submit a generic FAQ (i.e., an FAQ that applies to
multiple licensees) to the ROP WG. A generic FAQ may incorporate decisions made
from the white paper process.
At this point, the FAQ is called a “draft FAQ.” NEI typically provides the NRC with the
document containing the draft FAQs. This document is posted on the NRC’s public
Web site unless it contains plant-specific security information.

b.

Discussed. After a draft FAQ is introduced to the ROP WG, the WG will review and
discuss the FAQ to acquire understanding of assumptions and facts. The NRC may

Issue Date: 02/05/19

7

0608

also clarify the resident inspector’s position about the FAQ, if necessary. These
discussions may span over multiple ROP WG meetings. The content of draft FAQs
may be updated based on these discussions.
c.

Tentative Resolution. The ROP WG will develop a resolution to the FAQ, which will be
considered tentative. NRC staff will update the draft FAQ with a section titled,
“Tentative NRC Response,” which will document the NRC’s tentative position and a
proposed effective date.
The tentatively approved FAQ will remain tentative for a waiting period – normally until
the next regularly scheduled meeting – to allow a final opportunity for all stakeholders to
review the proposed FAQ resolution and provide any input. Stakeholders should
forward any feedback that impacts the resolution of the issue to the assigned lead
reviewer on the FAQ for resolution prior to the next scheduled ROP WG meeting. The
schedule for upcoming public meetings is posted on the NRC’s public Web site. After
stakeholders have had an opportunity to comment on a tentative resolution, the ROP
WG will determine whether the resolution can be considered final and approved.

d.

Appealed. If consensus on a resolution cannot be attained (typically by the second
ROP WG meeting after the FAQ is introduced), the NRR/DIRS Division Director will
determine the resolution, which will become the final approved resolution. The
NRR/DIRS Division Director will convey his decision at a public meeting (e.g., the ROP
WG meeting). Additional information about this process is provided in NEI 99-02.

e.

Approved. After a final resolution and effective date are determined, NRC staff will
update the draft FAQ with a section titled, “Final NRC Response,” that contains the
basis for the NRC’s resolution and an effective date, if necessary. The NRC will then
publish the final FAQ on its public Web site to characterize the FAQ as an “approved
FAQ,” unless the FAQ contains plant-specific security information. Approved FAQs are
treated as extensions of NEI 99-02 and become effective as of the effective date
specified in the NRC’s final response or as specified by NEI 99-02. The NRC will notify
internal and industry stakeholders that are not members of NEI of the status of the
FAQ. NEI is responsible for notifying industry stakeholders that are members of NEI of
the status of the FAQ.

f.

Withdrawn. The ROP WG may also decide to withdraw a draft FAQ; however, the
basis for the withdrawal and the status of the NRC deliberations should be documented
in the NRC’s response to the withdrawn FAQ for knowledge-transfer purposes.

g.

Archived. After the approved FAQs are incorporated into the next revision of NEI 9902, as applicable, the NRC will move the approved FAQs into the “archived FAQ” list on
the NRC’s public Web site. Withdrawn FAQs are also captured in the archived FAQ
list.

10.03 General Feedback about the PI Program. Anyone may provide feedback about the PI
Program. Such feedback can include clarifications of current guidance or suggested significant
changes to the PI Program (e.g., a new PI or a change to an existing PI). When the NRC
receives feedback about the PI Program, it evaluates the feedback to determine whether it has
merit and should be discussed at the ROP WG meetings. This section describes how various
stakeholders typically generate feedback about the PI program and how the feedback is
evaluated.

Issue Date: 02/05/19

8

0608

a.

Sources of Feedback. NRC staff can generate questions and feedback using a variety
of methods (e.g., ROP FBFs, surveys, self-assessment results, task force
recommendations, and sharing lessons learned through day-to-day interactions). If
NRC staff has specific suggestions for a new PI or for clarifying or modifying an existing
PI and associated guidance, the staff should submit an ROP FBF. The ROP FBF
should indicate “IMC 0308, Attachment 1” or “IMC 0308, Attachment 6” (if securityrelated) in the IP/IMC section of the ROP FBF. The staff may also provide such
feedback in ROP surveys of internal stakeholders. IRAB staff should consider
generating an ROP FBF to capture feedback coming from other internal sources of
information such as task force recommendations, Agency Action Review Meeting
results, or ROP survey feedback and comments.
The industry may generate white papers for proposed changes to NEI 99-02 that have
generic implications. Other stakeholders (e.g., the public, state/local governments, etc.)
can provide questions and feedback about the PI Program to the NRC’s Office of Public
Affairs (OPA). Methods for contacting OPA are listed on the NRC’s public Web site.
Stakeholders may also ask questions during the public ROP WG meetings and provide
feedback about the PI Program in ROP surveys of external stakeholders. IRAB staff
will generate an ROP FBF if the feedback warrants more detailed consideration for
program enhancements.
After IRAB staff receives suggestions to develop a new PI or to modify an existing PI,
the staff will evaluate the feedback to determine if it is possible or has merit. The staff
may involve NRC regional office staff and other technical staff as necessary. For
feedback from non-industry stakeholders, IRAB staff or technical leads in other NRC
offices in coordination with IRAB staff will generate a white paper to introduce the
feedback at an ROP WG meeting if the staff believes the feedback has merit. IRAB or
other technical staff may also discuss the issue with industry stakeholders at the ROP
WG meetings before deciding if the feedback has merit.

b.

White Paper Process. Stakeholders should introduce proposed generic changes to the
PI Program to the ROP WG via a draft white paper. White papers should contain the
following information, the extent of which can vary depending on the complexity of the
issue.
•
•
•
•

a description of the issue or circumstances that initiated the proposal
the proposal and its basis
the guidance that would be affected (e.g., NEI 99-02 sections, pages, and
lines)
implementation considerations (e.g., impact on information technology
support or infrastructure or an update to the OMB clearance for reporting
PIs)

The ROP WG should determine whether the white paper is proposing a clarification or
minor change to the guidance or if it is proposing a more significant change (e.g., a new
PI or a change in threshold values). Section 09.03.c should be implemented for white
papers that potentially involve significant changes to the PI Program.
The outcome of the ROP WG deliberations and the basis for that outcome shall be
documented in a final revision of the white paper, which shall then be entered into the
NRC’s Agencywide Documents Access and Management System (ADAMS). If the

Issue Date: 02/05/19

9

0608

ROP WG decides to not implement the white paper proposal, the white paper shall be
closed out in accordance with Section 09.04 of this IMC. If the ROP WG decides to
implement the white paper proposal, the ROP WG (typically the industry) will develop
an FAQ to incorporate the changes into NEI 99-02.
A listing of white papers is available on the NRC’s public Web site. A white paper does
not constitute a final decision or NRC-approved guidance for PI reporting; rather, an
approved FAQ, which incorporates the outcome of the white paper process into NEI 9902, constitutes approved guidance for PI reporting.
c.

Significant Changes to the PI Program. This section establishes guidance for
considering and making significant changes to the PI program, such as a new PI or a
modification of an existing PI. The process described in this section can be modified as
needed. Some activities (e.g., informing NRC management, seeking stakeholder
feedback, evaluating policy implications, and determining the impact of the change on
OMB Clearance No. 3150-0195) should be performed as needed or on an ongoing
basis. Because commercial nuclear power plant licensees voluntarily report PI data to
the NRC, continual interaction with the ROP WG is needed throughout this process.
1.

Identification of Potential Significant Changes to PI Program. Various
circumstances (e.g., Commission direction or results of ROP realignment
analyses, ROP self-assessment activities performed in accordance with
IMC 0307, “Reactor Oversight Process Self-Assessment Program,” or task group
reports) can shape and influence ongoing efforts to improve the PI Program
and/or ROP oversight. As circumstances warrant, efforts to identify potential
changes or improvements may take the form of a simple analysis or a more
detailed, systematic evaluation (such as an ROP realignment exercise).
Therefore, a number of approaches to the analysis could have merit given the
unique confluence of circumstances that give rise to the inquiry.
If an assessment reveals a gap in oversight of an ROP cornerstone, or if an
existing PI is ineffective, consistently generates many FAQs, or has the potential
to be misleading or create unintended consequences, the development of a new
PI or the significant modification of an existing PI may be a viable option to
ensure oversight of ROP cornerstone attributes is appropriate.
Significant changes to an existing PI can include a change to its thresholds.
Thresholds may need to be adjusted based on lessons learned from experience
with individual PIs. Such adjustments are not intended to continually raise
licensee performance expectations, but rather they are intended to ensure that
the initial thresholds, some of which were established without the benefit of
actual industry performance data, are performing as intended. A significant
change to an existing PI may also be necessary for plants with unique design
features that create challenges for adhering to NEI 99-02.

2.

Documentation of Proposed Significant Changes. Proposed significant changes
to the PI program should be documented in a white paper. Section 09.03.b
describes the basic content of a white paper. The following information should
be included for a proposed new or modified PI, as applicable and to the extent
practicable.

Issue Date: 02/05/19

10

0608

•
•
•
•
•

purpose of the proposed new or modified PI
definition of the proposed new or modified PI
the reporting elements for the proposed new or modified PI
calculations for the proposed new or modified PI
thresholds for the proposed new or modified PI

The draft white paper should be modified and refined as additional information
and feedback become available throughout the process.
3.

Evaluation of Proposed Significant Changes. In 2010, the ROP WG developed a
list of traits or characteristics that should be considered to guide the development
of a new PI to the extent practicable (ADAMS Accession Nos. ML101180467,
ML101530479, and ML101800474). These traits include considerations used for
selecting the initial set of PIs that was established in SECY-99-007 and later
documented in IMC 0308, Attachment 1. These traits can also be considered for
the development of significant changes to an existing PI.
The following traits should be considered for developing a new PI or a significant
change to an existing PI to the extent practicable.
•
•
•
•
•
•
•
•

capable of being objectively measured
allows for the establishment of a risk-informed threshold to guide
NRC and licensee actions
provides a reasonable sample of performance in the area being
measured
represents a valid indication of performance in the area being
measured
represents a verifiable (auditable) indication of performance in the
area being measured
encourages appropriate NRC and licensee actions
provides sufficient time for the NRC and licensees to correct
declining performance prior to posing undue risk to public health and
safety
adheres to the overall objectives of the ROP (i.e., risk-informed,
objective, predictable, and understandable)

The ROP WG should consider whether the proposed change to the PI program
will provide information that is not currently being collected. The ROP WG
should also consider whether the proposed new or modified PI warrants changes
to the ROP Inspection Program or other aspects of the PI Program to eliminate
unnecessary overlap or to ensure adequate coverage of ROP cornerstone
attributes.
4.

Stakeholder Feedback. After a stakeholder has developed a proposed concept
for a new or modified PI and begun the evaluation process, the stakeholder
should discuss the proposal with the ROP WG to acquire other stakeholder
feedback to inform the evaluation. The ROP WG may form a sub-group that
includes technical experts or representatives of the affected licensees.
The ROP WG may decide to use available industry performance information to
evaluate the proposal against the traits described in the previous section.

Issue Date: 02/05/19

11

0608

If historical data are available, they may be collected and used in this effort. If
such data are not readily available, the ROP WG may decide to use the best
information available or hypothetical data. An expert panel can also be
assembled to identify appropriate thresholds.
5.

Recommendation to NRR/DIRS Division Director. After evaluation of stakeholder
feedback, NRC staff should provide a recommendation to the NRR/DIRS Division
Director on whether to proceed with pursuing the PI change. Developing new
PIs or making significant changes to existing PIs can require significant
resources or may have policy implications. After consideration of the safety
insights that could be gleaned from the proposed PI change and associated
implications, the NRR/DIRS Division Director will inform IRAB staff of whether the
proposed change is feasible.
For PI changes that the DIRS Division Director determines are not feasible, NRC
staff will suspend consideration of the proposed changes and will close the issue
in accordance Section 09.02.d of this IMC.

6.

Pilot Project. Upon approval from the DIRS Division Director to proceed with
evaluating the proposed change, the ROP WG will develop a pilot project or a
tabletop exercise, as necessary, to further evaluate the change against the traits
listed in Section 09.03.c.3 and determine the efficacy of the PI. The pilot project
should be conducted using a representative sample of plants to collect data.
These plants would continue to provide data in accordance with the current
revision of NEI 99-02. The pilot project should benchmark those data to further
inform the characteristics of the proposed change, such as its definition,
calculation, and thresholds.
When the pilot project or the tabletop exercise has been completed, the results
and lessons learned will be used to update the white paper evaluation. The NRC
will then provide an opportunity for the industry, public, and other stakeholders to
provide feedback. This feedback will be evaluated by the staff and may be used
to modify the proposal.

7.

Final Recommendation. After the pilot project is conducted and stakeholders
provide feedback, the staff will make its final recommendation to the NRR/DIRS
Division Director as to whether to proceed with the proposal. Upon the
NRR/DIRS Division Director’s decision, the staff will proceed with the following
step.

8.

Implementation. If the staff determines that the proposal will not be
implemented, the staff should close out the issue in accordance with
Section 09.04. If the proposal will be implemented, the following steps shall be
taken, as necessary.
•
•
•

Issue Date: 02/05/19

The ROP WG will generate a generic FAQ to incorporate the change
into NEI 99-02. Refer to Section 09.02 of this IMC.
NRC staff will issue a RIS to inform stakeholders of the PI change
and its reporting criteria.
NRC staff will revise OMB Clearance No. 3150-0195. Revising the
OMB clearance could take approximately nine months to complete.

12

0608

•
•
•

Early consideration should be given to the potential need for revising
the OMB clearance to ensure it will not significantly delay final PI
implementation.
NRC staff will update ROP documents affected by the change (e.g.,
Attachments 1 or 6 of IMC 0308, IMC 0608, IP 71150, or IP 71151).
NRC staff will update its Web sites to incorporate the change.
NRC staff will develop training for its inspectors.

10.04. Closure. NRC staff will respond to the originator of questions or feedback, if contact
information is available, after the issue is resolved. The format and timing of the NRC’s
response will depend on how the feedback was received and its complexity. If the question or
feedback was generated using the ROP FBF process, then the lead reviewer will notify the
originator of the final response in accordance with the guidance established in IMC 0801. If the
question or feedback was generated using the FAQ process, then the ROP WG will adhere to
current guidance in NEI 99-02 for documenting and publishing the final resolution to the NRC’s
public Web site. If the question or feedback was generated by a public stakeholder, then the
NRC will respond in written correspondence.
NRC staff should determine whether any ROP documents (e.g., IMC 0308, Attachments 1 or 6;
IP 71150; or IP 71151) and its Web sites should be updated as a result of clarifications of or
changes that are made to the PI Program. This will help ensure that the basis for the changes
is communicated clearly and captured for knowledge-transfer purposes. NRC staff should verify
that any revision of NEI 99-02 correctly incorporates the decisions made since the previous
revision. Additionally, the staff should review RIS 2000-08, “Voluntary Submission of
Performance Indicator Data,” and update it to ensure consistency and adequacy. NRC staff
should ensure that the ROP WG meeting summaries document the results of the staff’s reviews
of NEI 99-02 revisions. If the issue involved a proposed significant change to the PI Program
that was not implemented, the staff shall update IMC 0308, Attachment 1, Table 1, “PI Program
Aspects Considered but Not Used.”
The ROP WG meeting summaries, including handouts that do not contain plant-specific
security-related information, are made publicly available in ADAMS.

Issue Date: 02/05/19

13

0608

0608-11

REFERENCES

CFR
IMC 0305, “Operating Reactor Assessment Program”
IMC 0307, “Reactor Oversight Process Self-Assessment Program”
IMC 0308, “Reactor Oversight Process Basis Document”
IMC 0308, Attachment 1, “Technical Basis for Performance Indicators”
IMC 0308, Attachment 6, “Basis Document for Security Cornerstone of the Reactor Oversight
Process”
IMC 0350, “Oversight of Reactor Facilities in a Shutdown Condition Due to Significant
Performance and/or Operational Concerns”
IMC 0351, “Implementation of the Reactor Oversight Process at Reactor Facilities in an
Extended Shutdown Condition for Reasons Other Than Significant Performance Problems”
IMC 0612, Appendix B, “Issue Screening”
IMC 0801, “Inspection Program Feedback Process”
IP 71150, “Discrepant or Unreported Performance Indicator Data”
IP 71151, “Performance Indicator Verification”
MD 8.13, “Reactor Oversight Process”
NEI 99-02, ARegulatory Assessment Performance Indicator Guideline@
NRC Enforcement Policy
OMB Clearance No. 3150-0195, “Voluntary Reporting of Performance Indicators”
(ML12020A281)
RIS 2000-08, Revision 1, “Voluntary Submission of Performance Indicator Data”
SECY-99-007, “Recommendations for Reactor Oversight Process Improvements”

END

Issue Date: 02/05/19

14

0608

ATTACHMENT 1
FLOWCHART FOR ADDRESSING QUESTIONS AND FEEDBACK
RELATED TO ROP PERFORMANCE INDICATORS
FAQs
White papers

Public input
ROP FBFs
Self-assessments
Lessons learned
White papers

NEI
Tracking
System
ROP
Working Group
meeting

Significant
change to PI
Program?

NRC
Tracking
System

Yes

D

No

Existing
policy
or
previously
addressed?

Yes

No

To NRC/DIRS
Division
Director

Yes

3rd WG
meeting?

Respond
and
close out

END

Develop proposed
response and discuss at
ROP WG

B
C

(Next Page)

A

Agreement on
tentative
approval?

No

C

No

Issue
approved
response

Yes

B

Waiting period
for stakeholder
comments

Update
NRC Web site
&
NEI FAQ log
Changes?

B
Yes

No

Update ROP &
NEI guidance
as necessary
& document in
next revision

D

Issue Date: 02/05/19

Att 1-1

0608

FLOWCHART FOR ADDRESSING QUESTIONS AND FEEDBACK
RELATED TO ROP PERFORMANCE INDICATORS (CONTINUED)

A

Develop white
paper for proposed
PI change

Evaluate & modify
proposed PI
change

Solicit stakeholder
feedback

DIRS Division
Director approval to
continue?

No
D

(Previous Page)

Yes
Pilot results;
solicit feedback;
update evaluation

DIRS Division
Director approval to
implement?

No
D

(Previous Page)

Yes
Update OMB clearance,
issue RIS, provide
training, and
develop FAQ

C

Issue Date: 02/05/19

(Previous Page)

Att 1-2

0608

ATTACHMENT 2
Revision History of IMC 0608
Commitment
Tracking
Number

Accession
Number
Issue Date
Change Notice

Description of Change

N/A

ML1011270423
04/21/01
CN 01-012
ML1021190669
04/16/02
CN 02-017

IMC 0608 issued.

N/A

N/A

N/A

N/A

N/A

ML0435601021
2/01/04
CN 04-027
ML070360605
02/27/07
CN 07-007
ML12219A374
09/26/12
CN 12-022

ML19025A257
02/05/19
CN 19-005

Issue Date: 02/05/19

Revised to document exclusion of T/2 fault exposure time in SSU
PIs, add guidance on how to resolve technical issues that are not
covered by the PI program, and add guidance for when a licensee
disagrees with HQ’s resolution of a feedback form.
Revised to delete information related to the Physical Protection
Cornerstone to ensure that potentially useful information is not
provided to a possible adversary.
Delete SSU, add MSPI; update flow charts; add definitions

Significant rewrite of questions and feedback section and
flowchart. New guidance was added on the white paper process
and considerations for developing new PIs. Some background
information was removed because it was redundant to and
contradicted with IMC 0308. Clarified other portions of guidance.
Incorporated ROP FBF 0608-1622.
Document updated to reflect FAQ process changes for licensee’s
that are not members of NEI. Other minor editorial changes
made. Also, IMC 0040 format compliance changes.

Att 2-1

Description of
Required
Training and
Completion
Date

Comment Resolution
and Closed Feedback
Form Accession
Number (PreDecisional, Non-Public
Information)

N/A

N/A

N/A

ML12270A018
FBF 0608-1622.

N/A

N/A

0608