NERC Standards

Exhibit A
Proposed Reliability Standards IRO-018-1 and TOP-010-1

IRO-018-1 Reliability Standard

IRO-018-1 – Reliability Coordinator Real-time Reliability Monitoring and Analysis Capabilities

A. Introduction
1.

Title:

Reliability Coordinator Real-time Reliability Monitoring and Analysis
Capabilities

2.

Number:

IRO-018-1

3.

Purpose:

Establish requirements for Real-time monitoring and analysis
capabilities to support reliable System operations.

4.

Applicability:
4.1. Functional Entities:
4.1.1. Reliability Coordinators

5.

Effective Date: See Implementation Plan

B. Requirements and Measures
R1.

Each Reliability Coordinator shall implement an Operating Process or Operating
Procedure to address the quality of the Real-time data necessary to perform its Realtime monitoring and Real-time Assessments. The Operating Process or Operating
Procedure shall include: [Violation Risk Factor: Medium ] [Time Horizon: Real-time
Operations]
1.1. Criteria for evaluating the quality of Real-time data;
1.2. Provisions to indicate the quality of Real-time data to the System Operator; and
1.3. Actions to address Real-time data quality issues with the entity(ies) responsible
for providing the data when data quality affects Real-time Assessments.

M1. Each Reliability Coordinator shall have evidence it implemented its Operating Process
or Operating Procedure to address the quality of the Real-time data necessary to
perform its Real-time monitoring and Real-time Assessments. This evidence could
include, but is not limited to: 1) an Operating Process or Operating Procedure in
electronic or hard copy format meeting all provisions of Requirement R1; and 2)
evidence the Reliability Coordinator implemented the Operating Process or Operating
Procedure as called for in the Operating Process or Operating Procedure, such as
dated operator or supporting logs, dated checklists, voice recordings, voice
transcripts, or other evidence.
R2.

Each Reliability Coordinator shall implement an Operating Process or Operating
Procedure to address the quality of analysis used in its Real-time Assessments. The
Operating Process or Operating Procedure shall include: [Violation Risk Factor:
Medium] [Time Horizon: Real-time Operations]
2.1. Criteria for evaluating the quality of analysis used in its Real-time Assessments;
2.2. Provisions to indicate the quality of analysis used in its Real-time Assessments;
and

Page 1 of 9

IRO-018-1 – Reliability Coordinator Real-time Reliability Monitoring and Analysis Capabilities

2.3. Actions to address analysis quality issues affecting its Real-time Assessments.
M2. Each Reliability Coordinator shall have evidence it implemented its Operating Process
or Operating Procedure to address the quality of analysis used in its Real-time
Assessments as specified in Requirement R2. This evidence could include, but is not
limited to: 1) an Operating Process or Operating Procedure in electronic or hard copy
format meeting all provisions of Requirement R2; and 2) evidence the Reliability
Coordinator implemented the Operating Process or Operating Procedure as called for
in the Operating Process or Operating Procedure, such as dated operator logs, dated
checklists, voice recordings, voice transcripts, or other evidence.
R3.

Each Reliability Coordinator shall have an alarm process monitor that provides
notification(s) to its System Operators when a failure of its Real-time monitoring
alarm processor has occurred. [Violation Risk Factor: Medium] [Time Horizon: Realtime Operations]

M3. Each Reliability Coordinator shall have evidence of an alarm process monitor that
provides notification(s) to its System Operators when a failure of its Real-time
monitoring alarm processor has occurred. This evidence could include, but is not
limited to, operator logs, computer printouts, system specifications, or other
evidence.
C. Compliance
1.

Compliance Monitoring Process
1.1. Compliance Enforcement Authority:
“Compliance Enforcement Authority” means NERC or the Regional Entity, or any
entity as otherwise designated by an Applicable Governmental Authority, in
their respective roles of monitoring and/or enforcing compliance with
mandatory and enforceable Reliability Standards in their respective
jurisdictions.
1.2. Evidence Retention:
The following evidence retention period(s) identify the period of time an entity
is required to retain specific evidence to demonstrate compliance. For instances
where the evidence retention period specified below is shorter than the time
since the last audit, the Compliance Enforcement Authority may ask an entity to
provide other evidence to show it was compliant for the full-time period since
the last audit.
The Reliability Coordinator shall keep data or evidence to show compliance as
identified below unless directed by its Compliance Enforcement Authority to
retain specific evidence for a longer period of time as part of an investigation.
The Reliability Coordinator shall retain evidence of compliance for
Requirements R1 and R3 and Measures M1 and M3 for the current calendar
year and one previous calendar year, with the exception of operator logs and

Page 2 of 9

IRO-018-1 – Reliability Coordinator Real-time Reliability Monitoring and Analysis Capabilities

voice recordings which shall be retained for a minimum of 90 calendar days,
unless directed by its Compliance Enforcement Authority to retain specific
evidence for a longer period of time as part of an investigation.
The Reliability Coordinator shall retain evidence of compliance for Requirement
R2 and Measure M2 for a rolling 30-day period, unless directed by its
Compliance Enforcement Authority to retain specific evidence for a longer
period of time as part of an investigation.
If a Reliability Coordinator is found non-compliant it shall keep information
related to the non-compliance until mitigation is complete and approved or for
the time specified above, whichever is longer.
The Compliance Enforcement Authority shall keep the last audit records and all
requested and submitted subsequent audit records.
1.3. Compliance Monitoring and Enforcement Program
As defined in the NERC Rules of Procedure, “Compliance Monitoring and
Enforcement Program” refers to the identification of the processes used to
evaluate data or information for the purpose of assessing performance or
outcomes with the associated Reliability Standard.

Page 3 of 9

IRO-018-1 – Reliability Coordinator Real-time Reliability Monitoring and Analysis Capabilities

Violation Severity Levels
R#

Violation Severity Levels
Lower VSL

R1.

N/A

Moderate VSL

High VSL

Severe VSL

The Reliability Coordinator's
Operating Process or
Operating Procedure to
address the quality of the
Real-time data necessary to
perform its Real-time
monitoring and Real-time
Assessments did not include
one of the elements listed in
Part 1.1 through Part 1.3.

The Reliability Coordinator's
Operating Process or
Operating Procedure to
address the quality of the
Real-time data necessary to
perform its Real-time
monitoring and Real-time
Assessments did not include
two of the elements listed in
Part 1.1 through Part 1.3.

The Reliability Coordinator's
Operating Process or
Operating Procedure to
address the quality of the
Real-time data necessary to
perform its Real-time
monitoring and Real-time
Assessments did not include
any of the elements listed in
Part 1.1 through Part 1.3;
OR
The Reliability Coordinator
did not implement an
Operating Process or
Operating Procedure to
address the quality of the
Real-time data necessary to
perform its Real-time
monitoring and Real-time
Assessments.

R2.

N/A

The Reliability Coordinator's
Operating Process or
Operating Procedure to
address the quality of

The Reliability Coordinator's
Operating Process or
Operating Procedure to
address the quality of

The Reliability Coordinator's
Operating Process or
Operating Procedure to
address the quality of

Page 4 of 9

IRO-018-1 – Reliability Coordinator Real-time Reliability Monitoring and Analysis Capabilities

analysis used in its Real-time
Assessments did not include
one of the elements listed in
Part 2.1 through Part 2.3.

analysis used in its Real-time
Assessments did not include
two of the elements listed in
Part 2.1 through Part 2.3.

analysis used in its Real-time
Assessments did not include
any of the elements listed in
Part 2.1 through Part 2.3;
OR
The Reliability Coordinator
did not implement an
Operating Process or
Operating Procedure to
address the quality of
analysis used in its Real-time
Assessments.

R3.

N/A

N/A

The Reliability Coordinator
has an alarm process
monitor but the alarm
process monitor did not
provide a notification(s) to
its System Operators when a
failure of its Real-time
monitoring alarm processor
occurred.

The Reliability Coordinator
does not have an alarm
process monitor that
provides notification(s) to its
System Operators when a
failure of its Real-time
monitoring alarm processor
has occurred.

D. Regional Variances
None.
E. Associated Documents
•

Implementation Plan

Page 5 of 9

IRO-018-1 – Reliability Coordinator Real-time Reliability Monitoring and Analysis Capabilities

Version History
Version

Date

Action

Change Tracking

1

October 30,
3015

New standard developed in Project 2009-02 to
respond to recommendations in Real-time Best
Practices Task Force Report and FERC directives.

N/A

1

May 5, 2016

Adopted by the Board of Trustees.

New

Page 6 of 9

Supplemental Material
Guidelines and Technical Basis
Real-time monitoring, or monitoring the Bulk Electric System (BES) in Real-time, is a primary
function of Reliability Coordinators (RCs), Transmission Operators (TOPs), and Balancing
Authorities (BAs) as required by TOP and IRO Reliability Standards. As used in TOP and IRO
Reliability Standards, monitoring involves observing operating status and operating values in
Real-time for awareness of system conditions. Real-time monitoring may include the following
activities performed in Real-time:
•
•
•
•

Acquisition of operating data;
Display of operating data as needed for visualization of system conditions;
Audible or visual alerting when warranted by system conditions; and
Audible or visual alerting when monitoring and analysis capabilities degrade or become
unavailable.

Requirement R1
The RC uses a set of Real-time data identified in IRO-010-1a Requirement R1 and IRO-010-2
Requirement R1 to perform its Real-time monitoring and Real-time Assessments. Requirements
to perform monitoring and Real-time Assessments appear in other Reliability Standards.
The RC's Operating Process or Operating Procedure must contain criteria for evaluating the
quality of Real-time data as specified in proposed IRO-018-1 Requirement R1 Part 1.1. The
criteria support identification of applicable data quality issues, which may include:
• Data outside of a prescribed data range;
• Analog data not updated within a predetermined time period;
• Data entered manually to override telemetered information; or
• Data otherwise identified as invalid or suspect.
The Operating Process or Operating Procedure must include provisions for indicating the quality
of Real-time data to operating personnel. Descriptions of quality indicators such as display color
codes, data quality flags, or other such indicators as found in Real-time monitoring
specifications could be used.
Requirement R1 Part 1.3 specifies the RC shall include actions to address Real-time data quality
issues with the entity(ies) responsible for providing the data when data quality affects Realtime Assessments. Requirement R1 Part 1.3 is focused on addressing data point quality issues
affecting Real-time Assessments. Other data quality issues of a lower priority are addressed
according to an entity's operating practices and are not covered under Requirement R1 Part
1.3.
The RC's actions to address data quality issues are steps within existing authorities and
capabilities that provide awareness and enable the RC to meet its obligations for performing
the Real-time Assessment. Examples of actions to address data quality issues include, but are
not limited to, the following:
•

Notifying entities that provide Real-time data to the RC;
Page 7 of 9

Supplemental Material
•

Following processes established for resolving data conflicts as specified in IRO-010-1a,
IRO-010-2, or other applicable Reliability Standards;

•

Taking corrective actions on the RC's own data;

•

Changing data sources or other inputs so that the data quality issue no longer affects
the RC's Real-time Assessment; and

•

Inputting data manually and updating as necessary.

The Operating Process or Operating Procedure must clearly identify to operating personnel how
to determine the data that affects the quality of the Real-time Assessment so that effective
actions can be taken to address data quality issues in an appropriate timeframe.
Requirement R2
Requirement R2 ensures RCs have procedures to address issues related to the quality of the
analysis results used for Real-time Assessments. Requirements to perform Real-time
Assessments appear in other Reliability Standards. Examples of the types of analysis used in
Real-time Assessments include, as applicable, state estimation, Real-time Contingency analysis,
Stability analysis or other studies used for Real-time Assessments.
Examples of the types of criteria used to evaluate the quality of analysis used in Real-time
Assessments may include solution tolerances, mismatches with Real-time data, convergences,
etc.
The Operating Process or Operating Procedure must describe how the quality of analysis results
used in Real-time Assessment will be shown to operating personnel.
Requirement R3
Requirement R3 addresses recommendation S7 of the Real-time Best Practices Task Force
report concerning operator awareness of alarm availability.
An alarm process monitor could be an application within a Real-time monitoring system or it
could be a separate system. 'Heartbeat' or 'watchdog' monitors are examples of an alarm
process monitor. An alarm process monitor should be designed and implemented such that a
stall of the Real-time monitoring alarm processor does not cause a failure of the alarm process
monitor.

Page 8 of 9

Supplemental Material
Rationale
Rationale for Requirement R1: The Reliability Coordinator (RC) uses a set of Real-time data
identified in IRO-010-1a Requirement R1 and IRO-010-2 Requirement R1 to perform its Realtime monitoring and Real-time Assessments. Requirements to perform Real-time monitoring
and Real-time Assessments appear in other Reliability Standards.
The Operating Process or Operating Procedure must include provisions for indicating the quality
of Real-time data to operating personnel. Descriptions of quality indicators such as display color
codes, data quality flags, or other such indicators as found in Real-time monitoring
specifications could be used.
Requirement R1 Part 1.3 of this standard specifies the RC shall include actions to address Realtime data quality issues affecting its Real-time Assessments in its Operating Process or
Operating Procedure. Examples of actions to address Real-time data quality issues are provided
in the Guidelines and Technical Basis section. These actions could be the same as the process
used to resolve data conflicts required by IRO-010-2 Requirement R3 Part 3.2 provided that this
process addresses Real-time data quality issues.
The revision in Part 1.3 to address Real-time data quality issues when data quality affects Realtime Assessments clarifies the scope of data points that must be covered by the Operating
Process or Operating Procedure.
Rationale for Requirement R2: Requirement R2 ensures RCs have procedures to address issues
related to the quality of the analysis results used for Real-time Assessments. Requirements to
perform Real-time Assessments appear in other Reliability Standards. Examples of the types of
analysis used in Real-time Assessments include, as applicable, state estimation, Real-time
Contingency analysis, Stability analysis or other studies used for Real-time Assessments.
The Operating Process or Operating Procedure must include provisions for how the quality of
analysis results used in Real-time Assessment will be shown to operating personnel. Operating
personnel includes System Operators and staff responsible for supporting Real-time operations.
Rationale for Requirement R3: The requirement addresses recommendation S7 of the Realtime Best Practices Task Force report concerning operator awareness of alarm availability.
The requirement in Draft Two of the proposed standard has been revised for clarity by
removing the term independent. The alarm process monitor must be able to provide
notification of failure of the Real-time monitoring alarm processor. This capability could be
provided by an application within a Real-time monitoring system or by a separate component
used by the System Operator. The alarm process monitor must not fail with a simultaneous
failure of the Real-time monitoring alarm processor.

Page 9 of 9

TOP-010-1 Reliability Standard

TOP-010-1 – Real-time Reliability Monitoring and Analysis Capabilities

A. Introduction
1.

Title:

Real-time Reliability Monitoring and Analysis Capabilities

2.

Number:

TOP-010-1

3.

Purpose:

Establish requirements for Real-time monitoring and analysis
capabilities to support reliable System operations.

4.

Applicability:
4.1. Functional Entities:
4.1.1. Transmission Operators
4.1.2. Balancing Authorities

5.

Effective Date: See Implementation Plan

B. Requirements and Measures
R1.

Each Transmission Operator shall implement an Operating Process or Operating
Procedure to address the quality of the Real-time data necessary to perform its Realtime monitoring and Real-time Assessments. The Operating Process or Operating
Procedure shall include: [Violation Risk Factor: Medium] [Time Horizon: Real-time
Operations]
1.1. Criteria for evaluating the quality of Real-time data;
1.2. Provisions to indicate the quality of Real-time data to the System Operator; and
1.3. Actions to address Real-time data quality issues with the entity(ies) responsible
for providing the data when data quality affects Real-time Assessments.

M1. Each Transmission Operator shall have evidence that it implemented its Operating
Process or Operating Procedure to address the quality of the Real-time data necessary
to perform its Real-time monitoring and Real-time Assessments. This evidence could
include, but is not limited to: 1) an Operating Process or Operating Procedure in
electronic or hard copy format meeting all provisions of Requirement R1; and 2)
evidence the Transmission Operator implemented the Operating Process or Operating
Procedure as called for in the Operating Process or Operating Procedure, such as
dated operator logs, dated checklists, voice recordings, voice transcripts, or other
evidence.
R2.

Each Balancing Authority shall implement an Operating Process or Operating
Procedure to address the quality of the Real-time data necessary to perform its
analysis functions and Real-time monitoring. The Operating Process or Operating
Procedure shall include: [Violation Risk Factor: Medium] [Time Horizon: Real-time
Operations]
2.1. Criteria for evaluating the quality of Real-time data;
2.2. Provisions to indicate the quality of Real-time data to the System Operator; and
Page 1 of 11

TOP-010-1 – Real-time Reliability Monitoring and Analysis Capabilities

2.3. Actions to address Real-time data quality issues with the entity(ies) responsible
for providing the data when data quality affects its analysis functions.
M2. Each Balancing Authority shall have evidence that it implemented its Operating
Process or Operating Procedure to address the quality of the Real-time data necessary
to perform its analysis functions and Real-time monitoring. This evidence could
include, but is not limited to: 1) an Operating Process or Operating Procedure in
electronic or hard copy format meeting all provisions of Requirement R2; and 2)
evidence the Balancing Authority implemented the Operating Process or Operating
Procedure as called for in the Operating Process or Operating Procedure, such as
dated operator logs, dated checklists, voice recordings, voice transcripts, or other
evidence.
R3.

Each Transmission Operator shall implement an Operating Process or Operating
Procedure to address the quality of analysis used in its Real-time Assessments. The
Operating Process or Operating Procedure shall include: [Violation Risk Factor:
Medium] [Time Horizon: Real-time Operations]
3.1. Criteria for evaluating the quality of analysis used in its Real-time Assessments;
3.2. Provisions to indicate the quality of analysis used in its Real-time Assessments;
and
3.3. Actions to address analysis quality issues affecting its Real-time Assessments.

M3. Each Transmission Operator shall have evidence it implemented its Operating Process
or Operating Procedure to address the quality of analysis used in its Real-time
Assessments as specified in Requirement R3. This evidence could include, but is not
limited to: 1) an Operating Process or Operating Procedure in electronic or hard copy
format meeting all provisions of Requirement R3; and 2) evidence the Transmission
Operator implemented the Operating Process or Operating Procedure as called for in
the Operating Process or Operating Procedure, such as dated operator logs, dated
checklists, voice recordings, voice transcripts, or other evidence.
R4.

Each Transmission Operator and Balancing Authority shall have an alarm process
monitor that provides notification(s) to its System Operators when a failure of its
Real-time monitoring alarm processor has occurred. [Violation Risk Factor: Medium]
[Time Horizon: Real-time Operations]

M4. Each Transmission Operator and Balancing Authority shall have evidence of an alarm
process monitor that provides notification(s) to its System Operators when a failure of
its Real-time monitoring alarm processor has occurred. This evidence could include,
but is not limited to, operator logs, computer printouts, system specifications, or
other evidence.
C. Compliance
1.

Compliance Monitoring Process
1.1. Compliance Enforcement Authority:
Page 2 of 11

TOP-010-1 – Real-time Reliability Monitoring and Analysis Capabilities

“Compliance Enforcement Authority” means NERC or the Regional Entity, or any
entity as otherwise designated by an Applicable Governmental Authority, in
their respective roles of monitoring and/or enforcing compliance with
mandatory and enforceable Reliability Standards in their respective
jurisdictions.
1.2. Evidence Retention:
The following evidence retention period(s) identify the period of time an entity
is required to retain specific evidence to demonstrate compliance. For instances
where the evidence retention period specified below is shorter than the time
since the last audit, the Compliance Enforcement Authority may ask an entity to
provide other evidence to show it was compliant for the full-time period since
the last audit.
The applicable entity shall keep data or evidence to show compliance as
identified below unless directed by its Compliance Enforcement Authority to
retain specific evidence for a longer period of time as part of an investigation.
The applicable entity shall retain evidence of compliance for Requirements R1,
R2, and R4, and Measures M1, M2, and M4 for the current calendar year and
one previous calendar year, with the exception of operator logs and voice
recordings which shall be retained for a minimum of 90 calendar days, unless
directed by its Compliance Enforcement Authority to retain specific evidence for
a longer period of time as part of an investigation.
The Transmission Operator shall retain evidence of compliance for Requirement
R3 and Measure M3 for a rolling 30-day period, unless directed by its
Compliance Enforcement Authority to retain specific evidence for a longer
period of time as part of an investigation.
If an applicable entity is found non-compliant it shall keep information related
to the non-compliance until mitigation is complete and approved or for the time
specified above, whichever is longer.
1.3. Compliance Monitoring and Enforcement Program
As defined in the NERC Rules of Procedure, “Compliance Monitoring and
Enforcement Program” refers to the identification of the processes used to
evaluate data or information for the purpose of assessing performance or
outcomes with the associated Reliability Standard.

Page 3 of 11

TOP-010-1 – Real-time Reliability Monitoring and Analysis Capabilities

Violation Severity Levels
R#

Violation Severity Levels
Lower VSL

R1.

N/A

Moderate VSL

High VSL

Severe VSL

The Transmission Operator's
Operating Process or
Operating Procedure to
address the quality of the
Real-time data necessary to
perform its Real-time
monitoring and Real-time
Assessments did not include
one of the elements listed in
Part 1.1 through Part 1.3.

The Transmission Operator's
Operating Process or
Operating Procedure to
address the quality of the
Real-time data necessary to
perform its Real-time
monitoring and Real-time
Assessments did not include
two of the elements listed in
Part 1.1 through Part 1.3.

The Transmission Operator's
Operating Process or
Operating Procedure to
address the quality of the
Real-time data necessary to
perform its Real-time
monitoring and Real-time
Assessments did not include
any of the elements listed in
Part 1.1 through Part 1.3;
OR
The Transmission Operator
did not implement an
Operating Process or
Operating Procedure to
address the quality of the
Real-time data necessary to
perform its Real-time
monitoring and Real-time
Assessments.

R2.

N/A

The Balancing Authority's
Operating Process or
Operating Procedure to
address the quality of the

The Balancing Authority's
Operating Process or
Operating Procedure to
address the quality of the

The Balancing Authority's
Operating Process or
Operating Procedure to
address the quality of the

Page 4 of 11

TOP-010-1 – Real-time Reliability Monitoring and Analysis Capabilities

Real-time data necessary to
perform its analysis
functions and Real-time
monitoring did not include
one of the elements listed in
Part 2.1 through Part 2.3.

Real-time data necessary to
perform its analysis
functions and Real-time
monitoring did not include
two of the elements listed in
Part 2.1 through Part 2.3.

Real-time data necessary to
perform its analysis
functions and Real-time
monitoring did not include
any of the elements listed in
Part 2.1 through Part 2.3;
OR
The Balancing Authority did
not implement an Operating
Process or Operating
Procedure to address the
quality of the Real-time data
necessary to perform its
analysis functions and Realtime monitoring.

R3.

N/A

The Transmission Operator's
Operating Process or
Operating Procedure to
address the quality of
analysis used in its Real-time
Assessments did not include
one of the elements listed in
Part 3.1 through Part 3.3.

The Transmission Operator's
Operating Process or
Operating Procedure to
address the quality of
analysis used in its Real-time
Assessments did not include
two of the elements listed in
Part 3.1 through Part 3.3.

The Transmission Operator's
Operating Process or
Operating Procedure to
address the quality of
analysis used in its Real-time
Assessments did not include
any of the elements listed in
Part 3.1 through Part 3.3;
OR
The Transmission Operator
did not implement an
Operating Process or
Operating Procedure to
address the quality of

Page 5 of 11

TOP-010-1 – Real-time Reliability Monitoring and Analysis Capabilities

analysis used in its Real-time
Assessments.
R4.

N/A

N/A

The responsible entity has
an alarm process monitor
but the alarm process
monitor did not provide
notification(s) to its System
Operators when a failure of
its Real-time monitoring
alarm processor occurred.

The responsible entity does
not have an alarm process
monitor that provides
notification(s) to its System
Operators when a failure of
its Real-time monitoring
alarm processor has
occurred.

D. Regional Variances
None.
E. Associated Documents
•

Implementation Plan

Version History
Version

Date

Action

Change Tracking

1

October 30,
2015

New standard developed in Project 2009-02 to
respond to recommendations in Real-time Best
Practices Task Force Report and FERC directives.

N/A

1

May 5, 2016

Adopted by the Board of Trustees

New

Page 6 of 11

Supplemental Material
Guidelines and Technical Basis
Real-time monitoring, or monitoring the Bulk Electric System (BES) in Real-time, is a primary
function of Reliability Coordinators (RCs), Transmission Operators (TOPs), and Balancing
Authorities (BAs) as required by TOP and IRO Reliability Standards. As used in TOP and IRO
Reliability Standards, monitoring involves observing operating status and operating values in
Real-time for awareness of system conditions. Real-time monitoring may include the following
activities performed in Real-time:
•
•
•
•

Acquisition of operating data;
Display of operating data as needed for visualization of system conditions;
Audible or visual alerting when warranted by system conditions; and
Audible or visual alerting when monitoring and analysis capabilities degrade or become
unavailable.

Requirement R1
The TOP uses a set of Real-time data identified in TOP-003-3 Requirement R1 to perform its
Real-time monitoring and Real-time Assessments. Functional requirements to perform
monitoring and Real-time Assessments appear in other Reliability Standards.
The TOP's Operating Process or Operating Procedure must contain criteria for evaluating the
quality of Real-time data as specified in proposed TOP-010-1 Requirement R1 Part 1.1. The
criteria support identification of applicable data quality issues, which may include:
• Data outside of a prescribed data range;
• Analog data not updated within a predetermined time period;
• Data entered manually to override telemetered information; or
• Data otherwise identified as invalid or suspect.
The Operating Process or Operating Procedure must include provisions for indicating the quality
of Real-time data to operating personnel. Descriptions of quality indicators such as display color
codes, data quality flags, or other such indicators as found in Real-time monitoring
specifications could be used.
Requirement R1 Part 1.3 specifies the TOP shall include actions to address Real-time data
quality issues with the entity(ies) responsible for providing the data when data quality affects
Real-time Assessments. Requirement R1 Part 1.3 is focused on addressing data point quality
issues affecting Real-time Assessments. Other data quality issues of a lower priority are
addressed according to an entity's operating practices and are not covered under Requirement
R1 Part 1.3.
The TOP's actions to address data quality issues are steps within existing authorities and
capabilities that provide awareness and enable the TOP to meet its obligations for performing
the Real-time Assessment. Examples of actions to address data quality issues include, but are
not limited to, the following:

Page 7 of 11

Supplemental Material
•

Notifying entities that provide Real-time data to the TOP;

•

Following processes established for resolving data conflicts as specified in TOP-003-3, or
other applicable Reliability Standards;

•

Taking corrective actions on the TOP's own data;

•

Changing data sources or other inputs so that the data quality issue no longer affects
the TOP's Real-time Assessment; and

•

Inputting data manually and updating as necessary.

The Operating Process or Operating Procedure must clearly identify to operating personnel how
to determine the data that affects the quality of the Real-time Assessment so that effective
actions can be taken to address data quality issues in an appropriate timeframe.
Requirement R2
The BA uses a set of Real-time data identified in TOP-003-3 Requirement R2 to perform its
analysis functions and Real-time monitoring. Requirements to perform monitoring appear in
other Reliability Standards.
The BA's Operating Process or Operating Procedure must contain criteria for evaluating the
quality of Real-time data as specified in proposed TOP-010-1 Requirement R2 Part 2.1. The
criteria supports identification of applicable data quality issues, which may include:
• Data outside of a prescribed data range;
• Analog data not updated within a predetermined time period;
• Data entered manually to override telemetered information; or
• Data otherwise identified as invalid or suspect.
The Operating Process or Operating Procedure must include provisions for indicating the quality
of Real-time data to operating personnel. Descriptions of quality indicators such as display color
codes, data quality flags, or other such indicators as found in Real-time monitoring
specifications could be used.
Requirement R2 Part 2.3 specifies the BA shall include in its Operating Process or Operating
Procedure actions to address Real-time data quality issues when data quality affects its analysis
functions. Requirement R2 Part 2.3 is focused on addressing data point quality issues affecting
analysis functions. Other data quality issues of a lower priority are addressed according to an
entity's operating practices and are not covered under Requirement R2 Part 2.3.
The BA's actions to address data quality issues are steps within existing authorities and
capabilities that provide awareness and enable the BA to meet its obligations for performing its
analysis functions. Examples of actions to address data quality issues include, but are not
limited to, the following:
•

Notifying entities that provide Real-time data to the BA;

Page 8 of 11

Supplemental Material
•

Following processes established for resolving data conflicts as specified in TOP-003-3 or
other applicable Reliability Standards;

•

Taking corrective actions on the BA's own data;

•

Changing data sources or other inputs so that the data quality issue no longer affects
the BA's analysis functions; and

•

Inputting data manually and updating as necessary.

The Operating Process or Operating Procedure must clearly identify to operating personnel how
to determine the data that affects the analysis quality so that effective actions can be taken to
address data quality issues in an appropriate timeframe.
Requirement R3
Requirement R3 ensures TOPs have procedures to address issues related to the quality of the
analysis results used for Real-time Assessments. Requirements to perform Real-time
Assessments appear in other Reliability Standards. Examples of the types of analysis used in
Real-time Assessments may include, as applicable, state estimation, Real-time Contingency
analysis, Stability analysis or other studies used for Real-time Assessments.
Examples of the types of criteria used to evaluate the quality of analysis used in Real-time
Assessments may include solution tolerances, mismatches with Real-time data, convergences,
etc.
The Operating Process or Operating Procedure must describe how the quality of analysis results
used in Real-time Assessment will be shown to operating personnel.
Requirement R4
Requirement R4 addresses recommendation S7 of the Real-time Best Practices Task Force
report concerning operator awareness of alarm availability.
An alarm process monitor could be an application within a Real-time monitoring system or it
could be a separate system. 'Heartbeat' or 'watchdog' monitors are examples of an alarm
process monitor. An alarm process monitor should be designed and implemented such that a
stall of the Real-time monitoring alarm processor does not cause a failure of the alarm process
monitor.

Page 9 of 11

Supplemental Material
Rationale
Rationale for Requirement R1: The Transmission Operator (TOP) uses a set of Real-time data
identified in TOP-003-3 Requirement R1 to perform its Real-time monitoring and Real-time
Assessments. Functional requirements to perform Real-time monitoring and Real-time
Assessments appear in other Reliability Standards.
The Operating Process or Operating Procedure must include provisions for indicating the quality
of Real-time data to operating personnel. Descriptions of quality indicators such as display color
codes, data quality flags, or other such indicators as found in Real-time monitoring
specifications could be used.
Requirement R1 Part 1.3 of this standard specifies the TOP shall include actions to address Realtime data quality issues affecting its Real-time Assessments in its Operating Process or
Operating Procedure. Examples of actions to address Real-time data quality issues are provided
in the Guidelines and Technical Basis section. These actions could be the same as the process
used to resolve data conflicts required by TOP-003-3 Requirement R5 Part 5.2, provided that
this process addresses Real-time data quality issues.
The revision in Part 1.3 to address Real-time data quality issues when data quality affects Realtime Assessments clarifies the scope of data points that must be covered by the Operating
Process or Operating Procedure.
Rationale for Requirement R2: The Balancing Authority (BA) uses a set of Real-time data
identified in TOP-003-3 Requirement R2 to perform its analysis functions and Real-time
monitoring. Requirements to perform monitoring appear in other Reliability Standards.
The Operating Process or Operating Procedure must include provisions for indicating the quality
of Real-time data to operating personnel. Descriptions of quality indicators such as display color
codes, data quality flags, or other such indicators as found in Real-time monitoring
specifications could be used.
Requirement R2 Part 2.3 of this standard specifies the BA shall include actions to address Realtime data quality issues affecting its analysis functions in its Operating Process or Operating
Procedure. Examples of actions to address Real-time data quality issues are provided in the
Guidelines and Technical Basis section. These actions could be the same as the process to
resolve data conflicts required by TOP-003-3 Requirement R5 Part 5.2 provided that this
process addresses Real-time data quality issues.
The revision in Part 2.3 to address Real-time data quality issues when data quality affects its
analysis functions clarifies the scope of data points that must be covered by the Operating
Process or Operating Procedure.
Rationale for Requirement R3: Requirement R3 ensures TOPs have procedures to address
issues related to the quality of the analysis results used for Real-time Assessments.
Requirements to perform Real-time Assessments appear in other Reliability Standards.
Examples of the types of analysis used in Real-time Assessments include, as applicable, state

Page 10 of 11

Supplemental Material
estimation, Real-time Contingency analysis, Stability analysis or other studies used for Real-time
Assessments.
The Operating Process or Operating Procedure must include provisions for how the quality of
analysis results used in Real-time Assessment will be shown to operating personnel. Operating
personnel includes System Operators and staff responsible for supporting Real-time operations.
Rationale for Requirement R4: The requirement addresses recommendation S7 of the Realtime Best Practices Task Force report concerning operator awareness of alarm availability.
The requirement in Draft Two of the proposed standard has been revised for clarity by
removing the term independent. The alarm process monitor must be able to provide
notification of failure of the Real-time monitoring alarm processor. This capability could be
provided by an application within a Real-time monitoring system or by a separate component
used by the System Operator. The alarm process monitor must not fail with a simultaneous
failure of the Real-time monitoring alarm processor.

Page 11 of 11