Download:
pdf |
pdfISO/RTO PERFORMANCE METRICS
Commission Staff Report
AD10-5-000
Federal Energy Regulatory Commission
October 21, 2010
Docket No. AD10-5
Table of Contents
Section
Page
I. Background ...................................................................................................... 5
II. Notice of Filing and Responsive Pleadings ..................................................... 7
III.
Discussion .................................................................................................... 7
A. Procedural Issues ......................................................................................... 7
1. Comments ................................................................................................ 7
2. Response .................................................................................................. 9
B. Policy Issue ................................................................................................ 10
1. Comments .............................................................................................. 10
2. Response ................................................................................................ 12
C. Metrics Issues............................................................................................. 12
1. General Issues ........................................................................................ 12
a.
Comments ........................................................................................ 12
2. Response ................................................................................................ 15
3. Discussion of Individual Metrics ........................................................... 17
a.
National and Regional Reliability Standards Compliance Metrics . 17
i. ISO/RTO Performance Metric......................................................... 17
ii.
Comments .................................................................................... 17
iii. Response ...................................................................................... 18
b.
Dispatch Reliability Metrics ............................................................ 19
i. ISO/RTO Performance Metric......................................................... 19
ii.
Comments .................................................................................... 20
iii. Response ...................................................................................... 21
c.
Load Forecast Accuracy Metric....................................................... 21
i. ISO/RTO Performance Metric......................................................... 21
ii.
Comments .................................................................................... 22
iii. Response ...................................................................................... 22
d.
Wind Forecasting Accuracy Metric................................................. 23
i. ISO/RTO Performance Metric......................................................... 23
ii.
Comments .................................................................................... 23
iii. Response ...................................................................................... 23
e.
Unscheduled Flows Metric .............................................................. 24
f. Transmission Outage Coordination Metric ......................................... 24
i. ISO/RTO Performance Metrics ....................................................... 24
ii.
Response ...................................................................................... 25
g.
Long-Term Reliability Planning – Transmission Metric ................ 26
i. ISO/RTO Performance Metrics ....................................................... 26
ii.
Comments .................................................................................... 26
iii. Response ...................................................................................... 27
h.
Long-Term Reliability Planning – Resources Metric...................... 27
2
Docket No. AD10-5
i. ISO/RTO Performance Metrics ....................................................... 28
ii.
Comments .................................................................................... 28
iii. Response ...................................................................................... 29
i. Infrastructure Investment – Interconnection and Transmission Process
Metrics ....................................................................................................... 31
i. ISO/RTO Performance Metrics ....................................................... 31
j. Comments............................................................................................ 31
i. Response .......................................................................................... 31
k.
Special Protection Systems.............................................................. 31
l. Backup Facility.................................................................................... 32
i. ISO/RTO Performance Metric......................................................... 32
ii.
Comments .................................................................................... 32
iii. Response ...................................................................................... 32
m.
Market Competitiveness .................................................................. 32
i. ISO/RTO Performance Metrics ....................................................... 32
ii.
Comments .................................................................................... 33
iii. Response ...................................................................................... 34
n.
Market Pricing Metrics .................................................................... 34
i. ISO/RTO Performance Metrics ....................................................... 34
ii.
Comments .................................................................................... 35
iii. Response ...................................................................................... 35
iv. System Lambda............................................................................ 36
o.
Energy Market Price Convergence .................................................. 36
i. ISO/RTO Performance Metrics ....................................................... 36
ii.
Comment ...................................................................................... 36
iii. Response ...................................................................................... 36
p.
Congestion Management Metrics .................................................... 37
i. ISO/RTO Performance Metrics ....................................................... 37
ii.
Comments .................................................................................... 37
iii. Response ...................................................................................... 38
q.
Resource Availability Metric........................................................... 39
i. ISO/RTO Performance Metric......................................................... 39
ii.
Comments .................................................................................... 39
iii. Response ...................................................................................... 40
iv. Fuel Diversity............................................................................... 41
r. Renewables Metrics............................................................................. 41
i. ISO/RTO Performance Metrics ....................................................... 41
ii.
Comments .................................................................................... 42
iii. Response ...................................................................................... 42
s. Administrative Costs Metrics .............................................................. 43
i. ISO/RTO Performance Metrics ....................................................... 43
ii.
Comments .................................................................................... 43
iii. Response ...................................................................................... 44
3
Docket No. AD10-5
Customer Satisfaction Metric .............................................................. 45
i. ISO/RTO Performance Metrics ....................................................... 45
ii.
Comments .................................................................................... 45
iii. Response ...................................................................................... 46
iv. Billing Controls Metric ................................................................ 46
IV.
APPENDIX A ............................................................................................ 47
V. APPENDIX B ................................................................................................ 50
t.
4
Docket No. AD10-5
Federal Energy Regulatory Commission Staff Report on ISO/RTO
Performance Metrics
Consistent with the recommendations of the report issued by the
Government Accountability Office, FERC Could Take Additional Steps to Analyze
Regional Transmission Organizations’ Benefits and Performance, 1 and the
Federal Energy Regulatory Commission’s (Commission) FY2010-2014 Strategic
Plan, Commission Staff has worked with Independent System Operators (ISO),
Regional Transmission Organizations (RTO), stakeholders and other experts to
develop standardized measures or metrics designed to track the performance of
ISO/RTO operations and markets. The Chairman plans to use the metrics
developed by Commission Staff to submit a report to Congress that will explain
what the measures and reported performance metrics communicate about the
benefits of ISOs/RTOs and, where appropriate, to identify changes that may need
to be made to address any performance concerns.
The purpose of this Commission Staff Paper is to describe the performance
metrics that have been developed to track the performance of RTO/ISO operations
and markets and to outline the process that will be taken to complete the report.
Commission Staff appreciates the public comments that have been provided and is
taking them into account in developing the final list of metrics to be used in
tracking the performance of ISO/RTO operations and markets.
I.
Background
Responding to a request for an investigation into ISO/RTO costs, structure,
processes and operations, 2 the Government Accountability Office, in a September
1
U.S. Gov’t Accountability Office, FERC Could Take Additional Steps to
Analyze Regional Transmission Organizations’ Benefits and Performance (2008)
(Government Accountability Office Report).
2
This request was made on May 21, 2007, by Senator Joseph I. Lieberman,
Chairman, and Senator Susan M. Collins, Ranking Minority Member, of the U.S.
Senate Committee on Homeland Security and Governmental Affairs, in a letter to
5
Docket No. AD10-5
2008 Report to the U.S. Senate Committee on Homeland Security and
Governmental Affairs, recommended that the Chairman take the following
actions: (1) work with RTOs, stakeholders, and other experts to develop
standardized measures that track the performance of RTO operations and markets;
and (2) report the performance results to Congress and the public annually, while
also providing an interpretation of: (a) what the measures and reported
performance communicate about the benefits of RTOs and, where appropriate, (b)
changes that need to be made to address any performance concerns. 3 The
Government Accountability Office Report also suggested that performance
metrics be explored for non-ISOs/RTOs.
As noted in the Government Accountability Office report, ISO/RTO
performance has been an issue of interest to stakeholders and the public since the
inception of ISOs and RTOs. The Government Accountability Office and
stakeholders acknowledge that ISOs/RTOs provide a significant amount of
information that measures the ISO or RTOs’ performance, and the ISOs/RTOs
communicate their results regularly. But the information being provided is not
supplied in a standardized format that allows for a comprehensive perspective of
all ISOs/RTOs. The effort described in this paper is intended to address these
gaps.
The Performance Metrics effort is also part of the Commission’s Strategic
Plan, which includes a Metrics Initiative. The first step of that effort is to explore
and develop appropriate operational and financial metrics for ISOs/RTOs in fiscal
year 2010. The next steps in the Metrics Initiative are as follows: (1) explore and
develop appropriate operational and financial metrics for non-ISOs/RTO regions
in fiscal year 2011; (2) establish appropriate common metrics between ISOs/RTOs
and non-ISO/RTO regions in fiscal year 2012; (3) monitor implementation and
performance in fiscal year 2013; and (4) evaluate performance and seek changes
as necessary in fiscal year 2014.
To begin the process of developing ISO/RTO performance metrics,
Commission Staff initiated a process with the ISOs/RTOs, stakeholders and other
experts to develop a set of consensus performance measures. As a first step,
Commission Staff developed a broad range of metrics designed to track the
the U.S. Government Accountability Office. The letter expressed the Senators’
concern that ISOs/RTOs may not be living up to their full potential with respect to
improving efficiencies and reducing costs, and that they might not have adequate
incentives to minimize costs.
3
Government Accountability Office Report at 59.
6
Docket No. AD10-5
performance of ISOs/RTOs in three specific areas, i.e., reliability, markets and
organizational effectiveness. Next, Commission Staff met with the ISOs/RTOs to
discuss Commission Staff’s proposed metrics and to determine the ability of each
ISO/RTO to compute the proposed metrics. Commission Staff then held focused
outreach meetings with a variety of industry, consumer, and state regulatory
associations 4 to receive their input. As a follow-up to that outreach, Commission
Staff’s proposed performance metrics were noticed for public comment and reply
comment in Docket No. AD10-5-000 on February 3, 2010.
II.
Notice of Filing and Responsive Pleadings
Notice of the February 3, 2010 filing was published in the Federal Register,
75 Fed. Reg. 7581 (2010) with comments due on or before March 5, 2010 and
reply comments due on or before March 19, 2010. Fifty-nine parties filed
comments and reply comments. The parties filing comments are listed in
Appendix A.
III.
Discussion
A.
Procedural Issues
1.
Comments
Some commenters 5 argue that the process used by Commission Staff to
develop the ISO/RTO performance metrics inappropriately allowed ISOs/RTOs to
develop the proposed metrics. They fault Commission Staff for not starting the
process with their recommendations, or basing the proposed metrics on their
recommendations. For these reasons, these commenters urge the Commission to
start with a clean slate 6 and to develop, in conjunction with stakeholders, the
4
APPA, AWEA, EEI, ELCON, EPSA, NARUC, NASUCA, NECPUC and
NRECA. Appendix A to this paper includes a list of intervenors and commenters.
5
APPA/ELCON, EMCOS, Industrial Customers, NRECA, ODEC and
TAPS.
6
Comments of EMCOS at 12 (“For the foregoing reasons, EMCOS request
that the Commission reject the proposed RTO Performance Metrics that
accompanied its Notice in this proceeding. EMCOS further suggest that the
Commission undertake a further proceeding[.]”); Comments of NRECA at 3
(“NRECA concurs and believes that any metrics used to evaluate RTO/ISO
benefits and performance should be developed, in the first instance, through the
RTOs’/ISOs’ own stakeholder processes.”).
7
Docket No. AD10-5
metrics by which the ISOs/RTOs will be evaluated. 7 Public Systems endorses the
issuance of a Notice of Proposed Rulemaking by the Commission and NARUC
and DEMEC recommend that the Commission hold technical conferences.
Commenters 8 also assert that the Commission must obligate ISOs/RTOs to issue
reports and data, and that either ISOs/RTOs or the Commission must be obligated
to take steps to improve performance. EEI expresses its understanding that the
Commission has no plans to make the metrics a regulatory requirement; it asserts
that the metrics should be for informational purposes only and not used to assess
penalties. 9
With respect to the process going forward, some commenters 10 recommend
an ongoing process that allows stakeholders the opportunity for informed review
and input before the metrics are finalized or revised from time to time. Others 11
suggest that ISOs/RTOs be required to file plans for remediation of deficiencies in
their metrics reports, and that these plans should be open for comment. Public
Systems asserts that the Commission should be obligated to issue a report and then
to issue orders based on comments stating whether the metrics will be retained or
revised. Industrial Customers recommend that the Commission, on its own
initiative, take all necessary steps to improve ISO/RTO performance. Ohio
Consumer Counsel asserts that the Commission should provide a neutral
interpretation of the data reported.
7
Comments of APPA/ELCON at 8 (“To ensure due process, the
Commission must allow all interested parties an equivalent opportunity to
participate in the development of these metrics.”); Comments of Public Systems at
2 (“Public Systems urge that the Commission view the current commenting
opportunities as the commencement, rather than completion, of a process of
performance metric identification, review, and refinement.”).
8
APPA/ELCON, Industrial Customers, NEPOOL, New Jersey Rate
Counsel and Public Systems.
9
Duke, PHI Companies and National Grid also express their concern that
the metrics may be used for reliability infractions, and request clarification that the
metrics not be used for this purpose.
10
NEPOOL, NESCOE, Public Systems, SoCal Edison and TAPS.
11
APPA/ELCON, Industrial Customers, NECPUC, NESCOE and New
Jersey Rate Counsel.
8
Docket No. AD10-5
COMPETE argues for establishing non-ISO/RTO metrics now, rather than
next year, to more quickly enable better informed decisions.
2.
Response
As outlined above, Commission Staff initiated a process with ISOs/RTOs,
stakeholders and other experts to develop standardized performance measures. As
discussed below, Commission Staff is taking into account stakeholder
recommendations in developing the list of metrics that will be submitted by each
ISO/RTO to the Commission. Commission Staff considers this process to be an
interactive and transparent development of performance metrics, and does not
believe that any benefits from restarting the process would justify the attendant
delay in using the draft metrics to gather performance data. Commission Staff’s
process to develop metrics incorporated a wide range of ISO/RTO and stakeholder
perspectives and input.
Some commenters suggest that the Commission must obligate ISOs/RTOs
to issue reports and data, and that either ISOs/RTOs or the Commission must be
obligated to take steps to improve performance. These suggestions are more
appropriate to a rulemaking proceeding than to this proceeding, in which the
Chairman and Commission Staff are responding to Government Accountability
Office recommendations. It is appropriate to (and sufficient for) this proceeding
that the ISOs/RTOs are providing their performance data voluntarily.
Commission Staff clarifies for commenters that the Chairman plans to send
Congress a report in 2011 that responds to the Government Accountability Office
Report and includes the following information:
three to five years of historical ISO/RTO performance data, provided by the
ISOs and RTOs, on common and standardized metrics for the ISOs/RTOs
under the jurisdiction of the Commission; 12
an explanation of the metrics and an interpretation of what the measures
and reported performance communicate about the benefits of ISOs/RTOs;
where appropriate, a discussion of possible changes that need to be made to
address any performance concerns; and
a description of the metrics development process, including a description of
how the Commission Staff worked with ISOs/RTOs, stakeholders, and
12
These ISOs/RTOs are ISO New England, Inc., New York Independent
System Operator, Inc., PJM Interconnection, L.L.C., Midwest Independent System
Operator, Inc., Southwest Power Pool, Inc., and the California Independent
System Operator Corporation.
9
Docket No. AD10-5
other experts to develop the metrics. The report would commit to filing a
follow-up report in the following year.
Commission Staff also clarifies that in fiscal year 2011 it will initiate a
voluntary and collaborative process – similar to the process used with ISOs/RTOs
-- for developing performance metrics in non-ISO/RTO regions. Inasmuch as the
primary focus of the Government Accountability Office recommendations was on
ISO/RTO performance measures, Commission Staff allocated its resources to
developing performance measures for these first.
B.
Policy Issue
1.
Comments
Certain municipal power interests, industrial customers and state
commissions 13 question whether organized markets are a better market design than
traditional cost-of-service regulation and request detailed cost and revenue data
from each generator to test their hypothesis. For example, APPA/ELCON contend
that “[a] fundamental argument underlying restructuring of wholesale electricity
markets is that competition will be more effective than cost-based price regulation
in achieving just and reasonable prices and benefits for consumers. . . . [A]
primary measure of the efficacy of competition in a market is whether the price of
a good, through competition, is driven down to the marginal cost of producing that
good.” 14
APPA/ELCON, Consumer Coalition and DEMEC assert that it is necessary
to obtain revenue, cost and profit information on individual generators and
demand response resources, in recognition of the fact that the price-cost mark-up
data does not reflect data in all ISO/RTO markets, such as those for ancillary
services and capacity markets.
Other commenters 15 observe that the Government Accountability Office
report raises the question of whether the ISO/RTO construct has provided the
promised benefits to customers. They conclude that the only way to answer that
question is to compare costs under organized regional markets to costs that would
have occurred absent the organized markets.
13
APPA/ELCON, California PUC, EMCOS, Ohio PUC and Public
Systems.
14
Comments of APPA/ELCON at 13.
15
Industrial Customers, Maryland PSC and TANC.
10
Docket No. AD10-5
APPA/ELCON, California PUC, TAPS and Commissioner Tyrone
Christy 16 assert that organized regional energy markets may be allowing
participants to exercise potentially inappropriate market power, i.e., prices in
organized regional markets may be exceeding marginal costs, and therefore the
resulting rates are not just and reasonable. In supplemental comments,
APPA/ELCON provided return on equity data for unregulated generation
providers in the PJM Interconnection that shows a return on equity range of 12 to
31 percent for these suppliers. 17 APPA/ELCON considers these returns to be high
and a strong indicator of the need for further investigation to ensure rates are just
and reasonable.
Opposing these positions, the Federal Trade Commission considers an
evaluation of the justness and reasonableness of an ISO/RTO market design using
a single-clearing-price, market-based rate regime to be beyond the scope of the
performance metric initiative.
EPSA points out that it is a collateral attack on the legality of market-based
pricing to assume that the just and reasonable standard requires marginal cost
pricing. EPSA also cites to court precedent affirming that the Commission may
rely upon market-based prices in lieu of cost-of-service regulation where the
Commission has found that buyers and sellers lack market power. IPPNY and P3
support these comments. EPSA, MidAmerican and IRC note that the Commission
conducts triennial reviews to determine whether markets are sufficiently
competitive to support market-based rates, and these markets are monitored on a
daily basis by market monitors and the Commission’s Office of Enforcement.
EPSA and MidAmerican consider measuring a variety of generator-specific
cost and revenue information to be inappropriate since this information does not
reflect market success or ISO/RTO benefits. The ISO New England Market
Monitor opposes the gathering and publishing of data on the financial performance
of individual market participants, since these data are not good measures of market
performance. FirstEnergy considers the cost-of-service preference of
APPA/ELCON and other parties to be short-sighted since it will result in low
short-term prices to the detriment of future resource investment and sustainable
electric markets.
16
Vice Chairman, Pennsylvania PUC.
17
This data is provided in 2009 Financial Performance of Owners of
Unregulated Generation, issued in May 2010.
11
Docket No. AD10-5
2.
Response
Commission Staff considers the APPA/ELCON position to be a criticism of
the Commission’s ISO/RTO market design using a single-clearing-price marketbased rate policy, and not an ISO/RTO performance issue. ISOs/RTOs can only
implement the Commission-approved market-based rate design for their organized
markets. Therefore, it is appropriate to evaluate the performance of ISOs/RTOs
based on the market design that the Commission has approved. Commission Staff
agrees with the point made by the Federal Trade Commission in its reply
comments:
This proposal confuses policy performance with
organizational performance metrics. FERC’s use of a
single market-clearing price approach for
compensating generators – rather than a pay-as-bid
approach combined with plant-specific price controls –
reflects an energy policy decision. The results of that
policy decision are not under the control of RTOs.
Thus, using generator costs compared to revenues as a
performance metric for RTOs would hold those
organizations accountable for results that reflect FERC
policy choices. For this reason, we recommend that
FERC not adopt generator costs compared to revenues
as a performance metric for RTOs.” 18
At the same time, however, Commission Staff recognizes that generator
profits may be reflective of competitive market conditions; therefore, metrics on
the price-cost mark-up 19 and generator net revenues are being added, as discussed
in the Market Competitiveness Metrics discussion below.
C.
Metrics Issues
1.
General Issues
a.
Comments
Most commenters consider Commission Staff’s proposed performance
metrics to be a good start toward developing useful, meaningful metrics.
18
Federal Trade Commission Reply Comments at 8.
19
California PUC, EMCOS, and NRECA endorse this metric.
12
Docket No. AD10-5
NEPOOL notes that, for New England, much of the specific data and information
identified in the proposed metrics is already reported by ISO New England or
otherwise publicly available, as required by the NEPOOL Participants Agreement.
At the same time, a number of commenters 20 urge the Commission to reevaluate the purpose of developing metrics, particularly in terms of their ability to
provide meaningful indicators of performance. They assert that the only useful
metrics are those that measure activities over which the ISOs/RTOs have control,
or that represent key functions of ISOs/RTOs. These commenters consider a
number of the draft performance metrics to be inappropriate, since they are
influenced primarily by other market participants or are subject to a variety of
market forces, such as transmission outage coordination and generator availability.
In addition, Midwest ISO Transmission Owners and PHI Companies assert
that the quantitative information provided in the ISO/RTO Performance Metrics
will be meaningless without any context as to the underlying circumstances that
affected the metric. For this reason, NECPUC recommends that ISOs/RTOs be
required to submit a narrative on each metric that explains and interprets the data.
Many of these commenters urge the Commission to recognize regional differences
among ISOs/RTOs.
Finally, commenters raise concerns that Commission Staff’s proposed
performance metrics do not include important metrics or are not designed
appropriately to provide meaningful measures, as follows.
The Federal Trade Commission recommends a grid efficiency performance
metric to measure an RTO’s contribution to the effective functioning of the
broader power system through economic dispatch efficiency, cooperation
with interconnection-wide planning efforts and constructive work to resolve
seams issues. Similarly, NY Transmission Owners endorse measuring
ISO/RTO effectiveness in eliminating seams between markets and a
measure of the inefficiencies caused by unauthorized loop flows. The
Federal Trade Commission also asserts that there should be metrics on the
minimum characteristics and functions specified in Order No. 2000 21 in
20
Midwest ISO Transmission Owners, National Grid, NECPUC, NEPOOL
and United Illuminating.
21
Regional Transmission Organizations, Order No. 2000, FERC Stats. &
Regs. ¶ 31,089 (1999), order on reh’g, Order No. 2000-A, FERC Stats. & Regs. ¶
31,092 (2000), aff’s sub nom. Pub. Util. Dist. No. 1 of Snohomish County
Washington v. FERC, 272 F.3d 607 (D.C. Cir. 2001).
13
Docket No. AD10-5
order to avoid the worst problems that can arise from using incomplete
performance metrics.
Commenters 22 recommend that ISOs/RTOs be required to undertake costbenefit studies on their major functions, to determine whether the functions
can be justified and to ensure accountability for ISO/RTO projections.
NARUC endorses the development of a baseline by which to compare the
ISOs/RTOs to regions outside these organized markets.
The ISO New England Market Monitor recommends that the metrics
include the following information: capacity market prices, capacity market
quantities offered and cleared, capacity required, the amount of existing
capacity and the amount of capacity added and retired each year. The New
England Market Monitor asserts that this information is required to indicate
whether the market is supporting new investment and efficient turnover of
assets. Dominion endorses a similar list of metrics. Ohio Consumer
Counsel also supports capacity price metrics and OMS supports metrics on
the cost of capacity and the method of compensating capacity in each
ISO/RTO.
ITC asserts that ISO/RTO performance should be measured based on
whether the regional entity has established clear cost allocation policies for
new transmission. AWEA recommends a metric to measure the percent of
transmission projects that qualify for cost sharing.
Certain commenters 23 suggest metrics to measure a wide range of market
structure data such as virtual bid and offer prices, credit defaults, liquidity
of exchange-traded energy, dollars transacted in each market (energy,
ancillary services, FTRs, virtual transactions) and lists of ISO/RTO
products.
In response to comments on the metrics, IRC asserts that the primary
purpose of the metrics is to provide a common database for the collection of
metrics in a uniform format and that metrics are not intended to substitute for
22
EMCOS, New Jersey Rate Counsel, Ohio Consumer Counsel and TAPS.
23
Allegheny, APPA/ELCON, California DWR, DC Energy, NRECA, Ohio
PUC and SoCal Edison.
14
Docket No. AD10-5
review and adjudication of tariffs under sections 205 and 206 of the Federal Power
Act. 24
2.
Response
Based on the comments received, Commission Staff has revised the metrics
listed in the February 3, 2010 list of proposed performance metrics. The full list
of revised metrics (ISO/RTO Performance Metrics) is provided in Appendix B.
The ISO/RTO Performance Metrics discussed below measure ISOs/RTOs’
performance in terms of their ability to administer their markets efficiently and
ensure reliability. ISO/RTO operations and administration are designed to provide
benefits in an efficient manner that provides value to consumers. A number of
metrics are influenced by factors beyond the control of ISOs/RTOs; nonetheless,
they provide useful information on trends in the ISO/RTO markets that reflect on
ISO/RTO operations, as discussed with regard to the individual metrics, below.
Commission Staff is recommending that the ISO/RTO reports include
narrative explanations to accompany the ISO/RTO data on performance metrics on
a number of issues of concern to commenters. In these narratives Commission
Staff encourages the ISOs/RTOs to quantify their information to the extent data is
available and to provide information on metrics that the individual ISOs/RTOs are
finding useful to track, even if they are not on the list of metrics in Appendix B.
For example, PJM performs a perfect dispatch analysis to provide insights into
how its organized markets are performing. This type of quantification and
exploration of potential metrics will facilitate the evolution of the performance
metrics.
Commission Staff agrees with the Federal Trade Commission that
measurement of ISO/RTO performance should consider the regional entities’
efforts to constructively resolve interconnection-wide issues and seams issues.
Commission Staff expects that this information can be communicated most
effectively in a narrative discussion since there is not a standardized product from
these efforts that can be easily put into a metric. The metrics discussions that
follow address the performance of ISOs/RTOs in carrying out the other minimum
functions 25 specified in Order No. 2000 (excluding the open-access same-time
24
16 U.S.C. §§ 824d, 824e (2006).
25
The Order No. 2000 minimum functions are tariff administration and
design, congestion management, parallel path flow, ancillary services, open access
same-time information system, total transmission capability, available
15
Docket No. AD10-5
information system and available transmission capability calculation function,
which do not bear directly on ISO/RTO performance). 26 The ISO/RTO
Performance Metrics do not address the minimum RTO characteristics that are
prescribed in Order No. 2000 (independence, scope and configuration, operational
authority and short-term reliability) since these characteristics explain what the
RTOs are, and not what they do; as such, the minimum characteristics do not
provide information on the performance of ISOs/RTOs.
Commission Staff considers a number of other commenters’
recommendations to be beyond the scope of this metrics effort. For example,
recommendations for cost-benefit studies and baseline analyses would provide
analyses of interest to stakeholders. However, these analyses are intended to serve
other purposes, such as a benchmark for industry practices, that go beyond the
purpose of this effort. Other recommendations are measures of Commission
policies, rather than measures of ISO/RTO performance. These include ITC’s
recommendation for a metric on whether an ISO/RTO has established a clear cost
allocation policy for new transmission and AWEA’s proposed metric on cost
sharing.
Certain commenters recommend metrics that would be redundant of the
proposed metrics. For example, recommendations for capacity market metrics to
measure investment in capacity and capacity offers, bids and prices are intended to
measure resource adequacy performance, which is already being measured in the
Long-Term Reliability Planning metrics in the reserve margin metrics. While
Commission Staff will not include these additional metrics, Commission Staff
expects that the narrative discussion accompanying the reserve margin metric will
address investment and capacity market impacts on resource adequacy.
Finally, there are a number of recommendations for metrics that provide
market information, but have no relation to ISO/RTO performance, such as
recommendations for market structure data, virtual bid and offer prices, credit
defaults, liquidity of exchange-traded energy, dollars transacted in each market
(energy, ancillary services, FTRs, virtual transactions) and lists of ISO/RTO
products. Commission Staff does not consider this data relevant to ISO/RTO
performance metrics and therefore will not request the development of metrics
based on this information.
transmission capability, market monitoring, planning and expansion and
interregional coordination.
26
The parallel path flow function will be included in the narrative
explanation of interconnection-wide issues and seams issues.
16
Docket No. AD10-5
3.
Discussion of Individual Metrics
The three major categories of performance metrics are reliability, market
benefits and organizational effectiveness. The reliability performance metrics,
discussed first, were chosen to measure the reliability of day-to-day operations in
metrics such as compliance with national and regional reliability standards,
dispatch, forecasting and Special Protection Schemes as well as to measure longterm reliability in metrics such as long-term generation and transmission planning.
The market benefits metrics were chosen to measure the performance of
ISOs/RTOs based on market prices, congestion management costs and resource
availability and to measure the efficiency of ISO/RTO markets in price
convergence and competition metrics. Finally, the organization effectiveness
metrics were chosen to measure ISO/RTO performance in accomplishing their
objectives in a cost-effective manner that provides value to market participants.
The individual metrics are discussed in the order they are listed in
Appendix B.
a.
National and Regional Reliability Standards
Compliance Metrics
i.
ISO/RTO Performance Metric
This metric measures the number of violations of national and regional
reliability standards and provides additional information on how these violations
were reported (self-reported or reported in audits) and the severity of the
violations. It thereby indicates both the significance of the violation and the
effectiveness of self-reporting procedures in identifying violations in a timely
manner. This metric is intended to be a comprehensive measure of reliability
performance based on functions undertaken by ISOs/RTOs. 27 The metric also
includes unserved energy (or load shedding) caused by violations.
ii.
Comments
EEI contends that measuring reliability performance on the basis of
aggregate violations will not provide an accurate representation of system
performance and that it is not possible initially to determine whether an increase or
27
A full listing of the reliability standards is provided at
http://www.nerc.com/page.php?cid=2|20.
17
Docket No. AD10-5
decrease in self-reported violations is good or bad. EEI also observes that a selfreported violation metric may discourage self-reporting. EPSA asserts that
violation data should be supporting information only since increases/decreases in
these measures are not indications of performance.
The California PUC recommends that ISOs/RTOs report the number of
violations regarding the NERC standard on operating reserve requirements.
NRECA recommends that this performance measure include megawatt-hours of
energy, and the number of customers, unserved due to violations. OMS states that
making an explicit recognition of the difference in ISO/RTO size would be
beneficial for the interpretation of relative ISO/RTO performance.
iii.
Response
Responding to commenters that question whether these metrics are accurate
representations of system performance, Commission Staff considers the metrics to
be basic measures of reliability in ISOs/RTOs and therefore they must be included
in a report on ISO/RTO operating performance. 28 Commission Staff considers
this information to be the basic building block from which a meaningful
assessment of ISO/RTO reliability performance must start. Responding to EEI’s
comments on self-reporting violations, Commission Staff notes that self-reported
violations can indicate an entity is improving its own compliance and catching
problems early. The Commission reaffirmed this in its Revised Policy Statement
on Penalty Guidelines. 29 Consistent with the Commission’s position on self-
28
Commission Staff expects, per the Commission’s regulations on the
enforcement of reliability standards, that the report would not identify any
nonpublic violations or any other non-public information related to cyber security
incidents or violations that could similarly jeopardize the bulk power system until
the information had been released by the Commission. See 18 C.F.R. § 39.7(b)(4)
(the disposition of each violation or alleged violation that relates to a
Cybersecurity Incident, or that would otherwise jeopardize the security of the Bulk
Power System if publicly disclosed shall be nonpublic unless the Commission
directs otherwise.)
29
See Enforcement of Statutes, Orders, Rules and Regulations, 132 FERC ¶
61,216 (2010) at P 141 (Self-reports, for example, add significant value to overall
industry compliance, and the Commission will continue to place great importance
on self-reporting. As we stated in the 2005 Policy Statement, “[c]ompanies are in
the best position to detect and correct violations of our orders, rules and
regulations, both inadvertent and intentional, and should be proactive in doing so.”
Providing credit for self-reporting gives organizations an incentive to detect and
18
Docket No. AD10-5
reporting, Staff does not expect that a self-reported violation metric will
discourage self-reporting. In addition, NERC already makes public filings to the
Commission of self-reported violations, so the total number of self-reported
violations will not be new information. RTOs will have the opportunity to explain
specific circumstances in the narrative accompanying the metrics.
Commission Staff agrees that the number of violations, alone, does not
provide meaningful information. Rather, the significance of the violations and the
context of their occurrence will be communicated in the narrative summaries that
will accompany the metrics. Based on comments received, metrics on compliance
with operating reserves standards and unserved energy (or load shedding) caused
by violations are included in the ISO/RTO Performance Metrics. Consistent with
the Revised Policy Statement on Penalty Guidelines, Commission Staffs
recommends that the unserved energy metric be calculated as the quantity of load
lost, in MWh, as a measure of the seriousness of the violation, rather than a
specific, individualized assessment of the value of losses of load that result from
reliability violations. 30 Responding to OMS, Commission Staff expects that the
narrative explanation that accompanies the metrics will provide the necessary
context as to the scale and configuration of each ISO/RTO to allow for a
meaningful interpretation of the metrics.
b.
Dispatch Reliability Metrics
i.
ISO/RTO Performance Metric
Dispatch reliability is measured by three metrics. The first metric measures
the performance of ISO/RTO dispatch operations in maintaining interconnection
steady-state frequency within defined limits by balancing power demand and
supply in real time. This is evaluated using Control Performance Standards 1 and
correct violations early. Self-reporting also assists the Commission’s review of
violations and facilitates the process of providing remedies to affected parties.)
30
Id. at P 4.
19
Docket No. AD10-5
2. 31 An alternative method of measurement is using the Balancing Authority Area
Control Error Limit. 32
The second metric measures the number of transmission loading relief or
unscheduled flow events that required ISO/RTO actions to manage transmission
flows in response to congestion for level three severity events and higher. This
metric indicates how often ISOs/RTOs must resort to manual actions to redirect
physical flows and therefore is an indication of the frequency of events that cannot
be managed through ISO/RTO dispatching and unit commitment.
The third metric measures the availability of energy management system
hardware and software, such as state estimators, for performing real-time
monitoring and security analysis functions. Availability is measured as the
percentage of hours that the energy management system is operationally available.
ii.
Comments
Duke considers the value of the transmission loading relief/unscheduled
flow metric to be questionable since ISOs/RTOs have different load profiles,
resources and other variables. For this reason, Duke contends that the
transmission loading relief/unscheduled flow metric must factor in different
operating practices and system configurations of ISOs/RTOs to be an effective
metric. Midwest ISO TOs and Mid-American also consider the transmission
loading relief/unscheduled flow metric to be of little value unless compared to the
number of similar events in regions with no organized markets. NEPOOL states
that the number of transmission loading relief events does not evaluate efficient
outcomes and may not be wholly applicable to New England
Midwest ISO TOs recommend that the transmission loading
relief/unscheduled flow metric also include information on the magnitude of
transmission loading reliefs and the amount of energy curtailed via transmission
31
Control Performance Standard 1 is a statistical measure of Area Control
Error variability. This standard measures Area Control Error in combination with
the Interconnection’s frequency error. It is based on an equation derived from
frequency-based statistical theory. Control Performance Standard 2 is a statistical
measure of Area Control Error magnitude. The standard is designed to limit a
control area’s unscheduled power flows.
32
Procedures for the Balancing Authority Area Control Error Limit
methodology can be found at
http://www.nerc.com/docs/standards/sar/Supporting_Documents.pdf.
20
Docket No. AD10-5
loading reliefs relative to the total amount of megawatt-hours transferred across an
ISO or RTO region. The Pennsylvania PUC also recommends that the metric
report transmission loading relief events by severity and category and separately
categorize transmission loading relief events called due to events within the
ISO/RTO and those called by outside coordinators. EPSA supports a metric to
measure energy management system availability.
iii.
Response
Commission Staff agrees with commenters that events requiring
transmission loading relief are a function of a wide range of factors outside
ISO/RTO control, such as system configuration, weather, and load shifts. The
purpose of ISO/RTO systems such as Security Constrained Economic Dispatch,
however, is to resolve system constraint problems and thereby to avoid reliance on
physical and manual procedures such as transmission loading relief that are more
costly, less efficient and less reliable. For example, Security Constrained
Economic Dispatch economically dispatches resources that have the greatest
impact on a constrained facility first and thereby minimizes out-of-merit dispatch
and the associated loss of system efficiency. Also, Security Constrained
Economic Dispatch reduces flows on constrained transmission lines much more
quickly and reliably than transmission load relief. For these reasons this metric,
with appropriate narrative explanations and evaluated over a number of years, will
provide an important indication of the trends of the effectiveness and efficiency of
each ISO/RTO’s operating performance. Commission Staff agrees with
commenters that the transmission loading relief/unscheduled flow metric can be
improved with severity information and energy curtailment data and for this
reason this information should be included in the metrics, to the extent the
information is available. For ISO New England, which does not use transmission
loading relief, a narrative explanation should be provided on its management of
transmission flows. Responding to EPSA, the ISO/RTO Performance Metrics
include an energy management system availability metric.
c.
Load Forecast Accuracy Metric
i.
ISO/RTO Performance Metric
Actual load as a percentage variance from forecasted load, for both peak
and off-peak periods, measures the effectiveness of the load forecasting function
of ISOs/RTOs. Since load forecasting provides the basis for resource
commitment, this metric impacts the incurrence of resource costs. The more
accurate an ISO/RTO is in forecasting load, the greater the likelihood that it can
commit sufficient resources in a cost effective manner that avoids overcommitment of resources, inefficient commitment of short lead-time resources or
21
Docket No. AD10-5
under-utilization of available resources. This metric measures the percentage
difference between actual load and forecasted load and the mean absolute
percentage error of day-ahead forecasts for all hours, the peak hour and the valley
hour. The mean absolute percentage error will be measured as the yearly average
error for all hours, the peak hour and the valley hour.
ii.
Comments
Midwest ISO TOs and Duke question whether cross-ISO/RTO comparisons
are meaningful because of regional differences. They also recommend: (1) that
the metric be designed to account for the use of interruptible and other demand
response resources during periods of high energy prices resulting in changes to
actual load figures, and (2) that the metric be limited to day-ahead forecasting and
thereby minimize the impact of weather variability. Ohio Consumer Counsel
requests clarification on the time-frames to be measured and recommends a metric
to measure the fees assessed for system imbalances caused by inaccurate load
forecasting. NECPUC recommends additional specificity as to whether the
forecasts are made on a weather-normalized basis. The New Jersey Rate Counsel
endorses metrics to measure long-term peak load forecast error, defined to be the
difference between the most recent forecast and the weather-normalized actual
peak load.
iii.
Response
Commission Staff clarifies for Midwest ISO TOs, Duke and Ohio
Consumer Counsel that the metric is based on the day-ahead forecast and that the
metric is calculated as the mean average of the absolute difference between the
forecasted load and the actual load divided by the forecasted load for all relevant
hours. Calculations will be made for the yearly average for all hours, the yearly
average for the peak hour (the highest load hour) of each day, and the yearly
average for the valley hour (the lowest load hour) of each day. The forecasts will
account for the impact of interruptible load and demand response resources.
Inasmuch as this metric is a day-ahead forecast measure, the measure minimizes
the impact of weather variability that may result in regional differences in forecast
accuracy. 33
Commission Staff understands New Jersey Rate Counsel’s concern to be
with the load forecasting process associated with resource adequacy planning.
Commission Staff expects that these long-term forecasts will be constantly shifting
33
We discuss the issue raised by Ohio Consumer Counsel of costs
associated with system imbalances in the Resource Availability section below.
22
Docket No. AD10-5
over time, to account for a number of factors such as the impact of economic
trends, new technologies and population shifts. Therefore, Commission Staff does
not consider comparisons of long-term forecasts or forecast errors to be
appropriate as an ISO/RTO performance metric. At the same time Commission
Staff recognizes that under-forecasting load can be a factor that leads to actual
reserve margins being below the forecast reserve margin, to the detriment of
reliability. To the extent under-forecasted load plays a role in achieving the target
reserve margin, this information will be addressed in the narrative explanation
accompanying the long-term generation reliability metric. 34
d.
Wind Forecasting Accuracy Metric
i.
ISO/RTO Performance Metric
This metric measures the percentage accuracy of actual wind availability
compared to forecasted wind availability as of the close of the prior day’s dayahead market. Improving the accuracy of the wind forecast will facilitate the
timely commitment and dispatch of sufficient supplemental resources.
ii.
Comments
California DWR asserts that this information should be a performance
metric and should be expanded to include solar, tidal and other intermittent
resources. California PUC recommends that the metric be standardized for
comparative purposes. National Grid endorses making this information a future
metric once wind monitoring and forecasting systems are in place. EEI considers
proposals to make this information into a performance metric to be premature
since the Commission is still investigating the benefits of gathering meteorological
data in Docket No. RM10-11. 35
iii.
Response
Responding to California DWR and California PUC, Commission Staff is
including wind forecasting as a standardized ISO/RTO Performance Metric in
recognition of the growing significance of this resource for nearly all of the
ISOs/RTOs. In recognition of the fact that forecasting for other variable energy
34
Information in the narrative on over-forecasting of load would also have
value. Over-forecasting can also be detrimental to ISO/RTO market efficiency
since it leads to over-procuring of capacity and higher than necessary costs.
35
Integration of Variable Energy Resources, 130 FERC ¶ 61,053 (2010).
23
Docket No. AD10-5
resources is either not performed by all ISOs/RTOs or is not performed according
to a standardized process that allows for comparison, Commission Staff will not
include the forecasts of other variable energy resources as performance metrics
and instead Commission Staff considers it appropriate that the narrative
discussions that accompany the metric include a discussion and quantification of
forecast accuracy for those variable energy resources that ISOs/RTOs forecast.
Responding to National Grid and EEI, Commission Staff considers it appropriate
to include wind forecasting accuracy as a metric since most ISOs/RTOs have wind
forecasting processes in place. Commission Staff does not find any basis to
conclude that the data and analysis developed in this report will impact the
Commission’s inquiry into eliminating barriers to the integration of variable
energy resources in Docket No. RM10-11. Our report simply reports the accuracy
of wind forecasts, whereas the inquiry has a different focus, namely to explore,
among other issues, various techniques to improve data gathering and forecasting
for variable energy resources.
e.
Unscheduled Flows Metric
Unscheduled flows are defined as the difference between net actual
interchange (actual power flow measured in real time) and the net scheduled
interchange. The two components of unscheduled flows are inadvertent energy,
defined to be the difference between actual and scheduled interchange for all
interties, and parallel flow (or loop flow), defined to be the actual power flow on a
contract path within an interconnection from one Balancing Authority Area to a
second Balancing Authority Area via “parallel” transmission lines through a third
Balancing Authority Area. 36 Parallel flows are a function of the interconnection’s
operating configuration, line resistance and physics. When unscheduled flows
exceed system operating limits, curtailments may occur and efficient scheduling of
the grid could be hindered. Accordingly, unscheduled flows provide information
relevant to operational planning that is part of a comprehensive assessment of the
reliability and efficiency of ISO/RTO markets. This metric is measured by the
absolute value of megawatts of unscheduled flows over a year and as a percentage
of total flows. No comments were submitted on unscheduled flows.
f.
Transmission Outage Coordination Metric
i.
ISO/RTO Performance Metrics
36
Parallel flows net out in an ISO/RTO region and therefore they do not
contribute to inadvertent interchange.
24
Docket No. AD10-5
Effective transmission outage coordination is defined as early notification
of planned outages of five days or longer – i.e., notification at least one month
prior to the outage commencement date – and timely review of outage impacts, per
ISO/RTO-established timeframes. Effective transmission outage coordination is
also measured by the percentage of planned outages that are cancelled due to
conflicting planned outages as well as forced (unscheduled) outages that could
cause reliability issues and additional congestion costs. 37 Effective transmission
outage coordination by ISOs/RTOs ensures that outages do not threaten system
reliability and that additional, and potentially more expensive, resources do not
need to be committed.
ii.
Comments
Some commenters 38 argue that this metric is not reasonably related to any
function that an ISO/RTO can perform because timely reporting of an outage is
under exclusive control of transmission owners. NEPOOL recommends that the
metric be modified to differentiate between reasons for outages, and suggests that
a better metric would be transmission outages that affect generation availability.
Midwest ISO TOs state that measuring an ISO/RTO response to planned outages
better demonstrates the effectiveness of ISO/RTO transmission outage
coordination. They also recommend revising the percentage of 200 kV and above
outages (both planned and unplanned) with less than two days’ notice by replacing
“unplanned” with “emergency” since “unplanned” is too broad a term. DC
Energy suggests measuring the one-month notification metric based on outages of
one day, instead of the proposed five day outage metric.
iii.
Response
Although transmission outage notices are the responsibility of transmission
owners, and not ISOs/RTOs, Commission Staff expects that this measure will
illuminate the effectiveness of coordination between transmission owners and
ISOs/RTOs. Commission Staff expects that the narrative discussion
accompanying the metric will identify the impact of transmission outages on
generation availability and on declared emergencies, as requested by NEPOOL
and Midwest ISO TOs. Responding to DC Energy, Commission Staff agrees that
outages of less than five days can be significant events that require advance
notification and therefore this information should be included. Accordingly,
37
The proposed metrics will measure outages for major transmission
facilities, which are defined, for purposes of the metrics, as 200 kV and higher.
38
CAISO, EEI, Mid-American, Midwest ISO and Midwest ISO TOs.
25
Docket No. AD10-5
Commission Staff would consider the addition of this information to be
appropriate when the ISOs/RTOs have collected the necessary information, for
possible inclusion in a follow-up report.
g.
Long-Term Reliability Planning – Transmission
Metric
i.
ISO/RTO Performance Metrics
This metric tracks the number of facilities approved to be constructed for
reliability purposes, the percentage of approved construction projects on schedule
and completed and the number of completed reliability and economic studies.
This information measures the effectiveness of the ISO/RTO expansion planning
process in identifying system reliability and economic needs in advance, thereby
providing sufficient time for market participants to develop either a market-based
solution (e.g., merchant transmission line, power plant or demand response) or a
regulated solution (e.g., transmission line) to system reliability and economic
requirements. The metric also measures the impact of demand response on the
type and cost of infrastructure investment.
ii.
Comments
Certain commenters 39 recommend additional metrics to measure project
costs. ITC recommends that this metric be developed based on all transmission
planning, not just reliability planning, and proposes a metric to measure the extent
to which efficient and innovative technologies are deployed. National Grid
recommends that the metric be based on transmission planning for both reliability
and economics.
Other commenters 40 assert that simply measuring the number of facilities
approved does not provide an accurate assessment of ISO/RTO effectiveness to
address reliability issues. TAPS recommends an additional metric for the status of
approved transmission construction projects. Commenters 41 also note that the
percentage of approved construction on schedule and completed is often beyond
the control of the ISO/RTO. NEPOOL states that the metric needs to distinguish
ISO/RTO performance from transmission owner performance. NECPUC notes
39
California DWR, NECPUC, NEPOOL and OMS.
40
Duke, EEI, EPSA and Midwest ISO TOs.
41
Duke, EPSA, Mid-American, Midwest ISO TOs and NECPUC.
26
Docket No. AD10-5
that ISOs/RTOs do not approve project construction, and therefore the metric
needs to be revised.
COMPETE recommends measuring the penetration of innovative resources
such as flywheels and battery storage. Ohio Consumer Counsel endorses a metric
on storage technologies.
iii.
Response
The primary purpose of this metric is to assess the extent to which
transmission solutions are deployed to meet reliability requirements, and therefore
the cost, congestion and innovative technology measures that commenters suggest
would go beyond the scope of the metric. With regard to OMS’s interest in a
congestion relief metric, the narrative discussion that accompanies the Congestion
Management metric, discussed below, will explain the impact of transmission
planning on congestion trends. Responding to ITC and National Grid, the third
metric listed in Appendix B measures both reliability and economic studies, and
therefore addresses all project planning, including economic projects. Considering
their growing importance in ISO/RTO operations, Commission Staff agrees with
commenters that information should be provided on the integration of innovative
technologies such as flywheels and battery storage into ISO/RTO markets in the
narrative explanations.
Commission Staff agrees with commenters that ISOs/RTOs should include
a narrative discussion with this metric that explains the significance of the metrics
data and provides a context to evaluate the benefit of ISO/RTO expansion
planning. Commission Staff also expects that the supplemental information
provided by ISOs/RTOs and the narrative discussion will address the status of
approved construction projects. While Commission Staff agrees with commenters
that the construction of planned projects is beyond the control of ISOs/RTOs,
Commission Staff notes that most of the metrics measure project approvals in the
expansion planning process 42 and the completion of expansion studies, all of
which are under the control of the ISOs/RTOs. Since project construction is the
end result of these processes, Commission Staff includes this metric as one of the
measures of the effectiveness of transmission planning.
h.
Long-Term Reliability Planning – Resources
Metric
42
We clarify for NECPUC that the project approval metrics measure the
number of reliability projects added to ISO/RTO expansion plans.
27
Docket No. AD10-5
i.
ISO/RTO Performance Metrics
Four metrics are employed to measure the effectiveness of long-term
reliability planning by ISOs/RTOs. The first metric, processing time for
generation interconnection requests, measures the effectiveness of ISO/RTO
processes in achieving timely interconnection of new resources that are needed to
ensure reliability. The second metric, actual reserve margins 43 compared to
planned reserve margins, measures the extent to which ISO/RTO generation
resource planning processes are ensuring long-term resource adequacy and
reliability. The third metric, the percentage of planned outages that are cancelled
by ISOs/RTOs, provides an indication of the effectiveness of ISO/RTO in
administering generation outage schedules. For example, a low cancellation
percentage indicates that generation owners were allowed to complete nearly all
the maintenance they had planned without incurring rescheduling costs or delays.
And the fourth metric, the number of generating units and their capacity under
reliability-must-run contracts, provides a measure of the degree to which an
ISO/RTO must depend on critical facilities to maintain reliability and the
flexibility of an ISO/RTO system to respond to emergencies and other
contingencies.
Two metrics measure the impact of demand response on long-term
reliability. 44 These are (1) the demand response share of total capacity, measured
in megawatts, and (2) the demand response share of ancillary services, measured
in megawatt-hours.
ii.
Comments
Dominion supports a metric that reports planned generation outages that are
cancelled or rescheduled after previous RTO approval, as a measure of the
effectiveness of ISO/RTO handling of generation outage schedules. EPSA
recommends a metric that measures the number of generating facilities and
megawatt capability of units retained under reliability must run contracts. The
Massachusetts AG endorses a measure of the megawatts procured in excess of the
regional installed capacity requirement and capacity costs incurred in excess of the
market clearing price. New Jersey Rate Counsel considers long-term peak load
data and forecasts to be an important input to resource adequacy plans, as well as
43
The reserve margin is the number of megawatts of resources available as
system reserves divided by the number of megawatts of peak load.
44
These metrics were originally listed in a separate demand response metric
in the proposed performance metrics.
28
Docket No. AD10-5
transmission and generation planning, and for this reason recommends including
metrics to measure long-term load forecast error. New Jersey Rate Counsel also
recommends metrics to measure deviations between actual reserve margins and
projected reserve margins, with discussion of reasons for deviations. To recognize
issues such as future carbon regulation and renewable portfolio standards, Ohio
Consumer Counsel recommends that ISOs/RTOs be required to report additional
information on the carbon contributions and generation fuels of ISO/RTO
generation portfolios, the intended use of the generation additions and the target
load or constraint the generation additions will serve. Ohio Consumer Counsel
also endorses a metric to assess the actual reserve margin from an economic
perspective using the projected value of lost load.
With respect to the demand response metrics, EPSA supports a metric to
measure the impact of demand response in peak conditions. Steel Producers
support a metric to measure the impact of demand response in providing ancillary
services. Steel Producers also argue for a more specific breakdown of the types of
demand response used in each ISO/RTO to include categories such as regulation
service, and for more refined metrics on the activity of these resources.
Other commenters 45 consider these metrics to be beyond ISO/RTO control
and therefore not suitable as performance metrics. NESCOE notes that these
metrics are developed within the states’ energy and environmental planning and
policy frameworks. EEI and NEPOOL assert that the metrics are not meaningful
since there are differences in resource mix, fuel prices, congestion and demand
response efficiency capability in each ISO/RTO.
iii.
Response
Commission Staff notes that the generation outage and reliability-must-run
metrics supported by Dominion and EPSA are included in the metrics. Since the
purpose of the metric is to measure reliability impacts, Commission Staff does not
consider the capacity market efficiency and cost metrics proposed by the
Massachusetts AG to be appropriate. At the same time, Commission Staff agrees
with the Massachusetts AG that the efficiency of ISO/RTO capacity markets
should be evaluated in a performance review. Since several ISOs/RTOs are not
required to purchase capacity for their region and therefore these efficiency
measures do not reflect their performance, a performance metric would not be
appropriate. For those ISOs/RTOs that do perform this function, this capacity
efficiency and cost information can be discussed in the narrative explanation that
accompanies the total power cost metric, discussed below, for the capacity cost
45
Allegheny, Midwest ISO TOs, and NESCOE.
29
Docket No. AD10-5
component. Commission Staff agrees with New Jersey Rate Counsel that reserve
margin estimates are important ISO/RTO functions that should be evaluated as
part of an ISO/RTO performance review. The primary focus of this metric,
however, is to measure the performance of ISOs/RTOs in achieving resource
adequacy. Accordingly, Commission Staff agrees that it is appropriate that there
be a discussion of a number of factors that impact the adequacy of the actual
reserve margin vis-à-vis the projected reserve margin, including load forecasts and
energy efficiency trends, in the narrative explanation that accompanies the metric,
rather than creating an additional metric. 46 Commission Staff also notes that
reserve margin analysis varies from one ISO/RTO to another, as New Jersey Rate
Counsel also recognizes, and therefore metric information would not be
comparable across ISOs/RTOs.
The additional information regarding fuel mix and carbon emissions that
Ohio Consumer Counsel recommends collecting does not measure ISO/RTO
performance with respect to long-term reliability planning. The metrics relating to
fuel diversity, demand response, and renewables, however, will provide
information relevant to the issues of concern to Ohio Consumer Counsel, as
discussed below. Commission Staff does not agree with the recommendation to
require information on the value of lost load since Commission Staff does not
consider this information to be pertinent to measuring the long-term reliability
planning performance of ISOs/RTOs.
Regarding the demand response metrics, Commission Staff recognizes that
the amount of demand response in a market is not under the direct control of
ISOs/RTOs, and that they are influenced by a number of factors, such as state
demand response initiatives. Nonetheless, these metrics may be useful indicators
of the impact on long-term reliability planning. Commission Staff recognizes that
regional differences among the ISOs/RTOs will result in differing integration
results for demand response resources.
With respect to comments requesting more granularity in the data and more
refined metrics, Commission Staff considers demand response data for separate
ancillary services markets to be appropriate, as recommended by commenters, to
the extent it is available.
46
We note that we also consider it appropriate to include information on the
load forecasting process for resource adequacy in the narrative explanation, as
discussed in the Load Forecasting Accuracy section above.
30
Docket No. AD10-5
i.
Infrastructure Investment – Interconnection and
Transmission Process Metrics
i.
ISO/RTO Performance Metrics
These metrics track the progress made by ISOs/RTOs in completing their
reliability reviews – namely, feasibility, system impact and facility studies – of
interconnection and transmission service requests in a timely and efficient manner.
The metrics track the number of requests, the time required to complete the
reliability reviews and the costs of completing each of the three types of studies.
ii.
Comments
ITC and EPSA support this performance metric. California PUC
recommends that the information be clarified and normalized to facilitate
benchmarking or comparative assessments. Ohio Consumer Counsel endorses
reporting the information by utility and by independent power provider as well as
providing information on formal complaints filed. Duke asserts that the
interconnection and transmission service requests must be differentiated.
iii.
Response
Responding to California PUC and Ohio Consumer Counsel, Commission
Staff expects that the narrative explanation that accompanies this metric will
address why there are differences between ISOs/RTOs in the completion and
timing of transmission studies, as well as information on the types of entities being
interconnected and the status of formal complaints. While Commission Staff
agrees with Duke that the information would be more meaningful if it had separate
information for interconnections and transmission service requests, this
information is not available from the ISOs/RTOs. To address Duke’s concern,
Commission Staff would consider it appropriate that the report provide narrative
explanations on trends in interconnections and transmission service requests.
j.
Special Protection Systems
Special Protection Systems 47 are automatic protection systems designed to
detect abnormal or predetermined system conditions and take corrective actions
such as changing demand, generation or system configurations in order to
47
Special Protection Systems are also referred to as Special Protection
Schemes, Remedial Action Schemes (RAS) or System Integrity Protection
Schemes (SIPS).
31
Docket No. AD10-5
maintain system stability, acceptable voltage levels or power flows. The metrics
measure both the frequency with which the ISO/RTO relies on these systems and
their effectiveness, as measured by successful activations and the number of
unintended activations. Duke considers this information to be more reflective of
operations of transmission owners and not ISOs/RTOs. While Commission Staff
agrees that Special Protection Systems are actions undertaken by transmission
owners, these actions must be coordinated with ISO/RTO operators. Accordingly,
this metric is a measure of ISO/RTO-wide performance in managing system
stability, voltage levels and power flows and therefore it is appropriate that this
information is included in an assessment of ISO/RTO reliability performance.
k.
Backup Facility
i.
ISO/RTO Performance Metric
This metric measures the capability of the ISO/RTO to continue reliability
operations in the event the primary control center is not functioning. The metric
includes information on readiness training, communications capabilities and quick
response capabilities.
ii.
Comments
National Grid recommends that this information be deleted since similar
NERC requirements already exist. EEI notes that this information is the same as
the requirements of NERC EOP-008, and therefore its purpose is questionable.
Duke asserts that NERC standards do not require that backup facilities be staffed
at all times, and therefore metrics should be revised to report on whether the
facility can be staffed in a short amount of time.
iii.
Response
Upon further consideration, Commission Staff concludes that this
information is sensitive in nature. It includes procedures and information that are
critical to maintaining the integrity of the electric gird and for this reason
recommends not requiring that ISOs/RTOs release this information to the public.
Therefore, Commission Staff deletes this metric.
l.
Market Competitiveness
i.
ISO/RTO Performance Metrics
By design, the Security Constrained Economic Dispatch and Security
Constrained Unit Commitment processes in ISO/RTO markets are intended to
32
Docket No. AD10-5
foster competition among suppliers and to maximize production for the least-cost
suppliers per least-cost unit commitment, thereby benefiting consumers. The
following metrics are intended to evaluate how well ISO/RTO markets are
performing in achieving competitive results:
The first metric, the price-cost mark-up, compares the system
marginal price to the system marginal cost, 48 assuming no system
constraints. The difference between the marginal price and marginal
cost indicates the degree of competition in ISO/RTO markets.
ISO/RTO markets are more competitive the closer prices are to
marginal costs. This metric is measured as the percentage mark-up
for each year.
The second metric, generator net revenue, measures the revenue that
a new generator would earn above its variable production costs if it
were to operate only when its variable production costs were less
than the energy price. This metric can be an indicator of whether
generator net revenues are sufficient to ensure new investment, if
needed, and are consistent with competitive markets. This metric is
measured on an annual basis.
The third metric provides an indication of the magnitude of
mitigation occurring in ISO/RTO markets, as measured by the
dollars and megawatt hours mitigated, the percentage of dollars and
megawatt hours mitigated and the percentage of unit hours prices are
capped on an annual basis. This metric will be supplemented by a
discussion of the significance of the actions for market power and
competition.
ii.
Comments
These metrics are endorsed by a number of commenters. 49 Public Systems
and Ohio Consumer Counsel recommend a break-down of price-cost mark-up data
48
To clarify the difference between this metric and the System Lambda
metric discussed below, the price-cost mark-up compares the difference between
marginal prices and marginal costs, whereas system lambda only measures
marginal cost.
49
California DWR, California PUC, DEMEC, EMCOS, EPSA,
Massachusetts AG, NECPUC, NEPOOL, NRECA, New York PSC and NY
Transmission Owners
33
Docket No. AD10-5
by fuel type, asset class and sub-market. Ohio Consumer Counsel and OMS also
recommend metrics on market concentration and market power. California PUC,
California DWR and the Missouri Commission recommend metrics to measure
price volatility.
iii.
Response
Commission Staff will not request the break-down of price-cost mark-up
data by fuel type, asset class or sub-market, as Public Systems and Ohio Consumer
Counsel recommend. In ISO/RTO markets, the price that clears the market
applies to all types of resources offering into the market at the price node, so there
is no price applicable only to a fuel type or asset class. With respect to sub-market
competition, the report should include a narrative discussion accompanying the
metric that will evaluate competitive issues in sub-markets. The narrative
discussion should also address market concentration and market power to the
extent they have a bearing on market competitiveness, and therefore additional
metrics recommended by Ohio Consumer Counsel and OMS are not needed. With
respect to the price volatility measures recommended by California PUC,
California DWR and the Missouri Commission, the narrative discussion should
address those price movements that reflect anti-competitive behavior.
m.
Market Pricing Metrics
i.
ISO/RTO Performance Metrics
The four market pricing metrics measure the customer cost impact of
ISO/RTO markets. The first measure, the load-weighted locational marginal price
metric, measures the cost to load of energy purchased in ISO/RTO markets. The
second metric, components of total power costs, breaks out each element of all
costs paid by load, thereby providing a comprehensive assessment of all ISO/RTO
market costs. 50 The third measure, the load-weighted, fuel-adjusted locational
marginal price, is derived by holding fuel costs constant over a defined time period
to show the trend of non-fuel customer costs over this period. This metric isolates
the customer cost impact of cost elements such as transmission costs, congestion
and losses, thereby providing a measure of the effectiveness of ISO/RTO market
management. This metric also reflects the impact of load growth, investments in
resources and the retirement of uneconomic facilities, and therefore it measures
factors that are not entirely within the control of ISOs/RTOs. The fourth metric
50
The cost break-down includes the following cost categories: ISO/RTO
costs and regulatory fees, operating reserve costs, ancillary services costs,
transmission costs, capacity costs and energy costs.
34
Docket No. AD10-5
measures the impact of demand response on market prices, including impacts
associated with voluntary curtailments by demand response during heat waves and
other emergency conditions. All four metrics are measured on an annual basis.
ii.
Comments
Certain commenters 51 recommend measuring additional costs, such as the
cost of reliability-must-run units, reliability unit commitment charges and
payments, high voltage access charges, excess cost payments, default interest
charges, lost opportunity costs and FERC annual charges. Commenters 52 also
argue for greater granularity in the reported costs and the time periods analyzed, as
well as an analysis of factors driving costs. The ISO New England Market
Monitor considers the fuel-adjusted metric to be of limited usefulness since there
is no standard methodology to calculate the fuel-adjusted price and it is not
possible to capture shifts in relative costs of fuels.
iii.
Response
The cost categories covered by the total power cost metric include the
major costs of ISO/RTO markets to consumers, and therefore Commission Staff
considers this metric to be a comprehensive measure of the costs of ISO/RTO
markets. Commission Staff believes that the narrative explanation of the causes of
cost trends that accompany each ISO/RTO metrics submission will be responsive
to the concerns of commenters that recommend more granularity in the cost
categories, down to the level of costs for individual ISO/RTO services so that the
root causes of cost trends can be better identified. The narrative discussion should
also provide information on peak price trends, as requested by OMS.
Responding to commenters’ interest in tracking Commission annual
charges, Commission Staff considers these costs to be beyond the control of
ISOs/RTOs and therefore do not reflect ISO/RTO performance. With respect to
the fuel-adjusted cost metric concern raised by the ISO New England Market
Monitor, Commission Staff recognizes that each ISO/RTO uses a different year as
its starting point for holding fuel prices constant and the ISOs/RTOs have differing
fuel mixes, making comparisons across ISOs/RTOs difficult. Nonetheless,
analysis of long-term trends is a useful measure of the effectiveness of ISOs/RTOs
51
California PUC, DC Energy, Dominion, NECPUC, NEPOOL, SoCal
Edison and Westar.
52
California PUC, ISO New England Market Monitor, New Jersey Rate
Counsel and OMS.
35
Docket No. AD10-5
in managing non-fuel costs, and therefore the metric will provide useful
performance information.
Commission Staff notes that a fourth metric has been added to the proposed
list of metrics provided in the February 3, 2010 notice to measure the impact of
demand response on market prices. The purpose of this metric is to ensure the
Market Pricing analysis addresses the impact of integrating demand response in
ISO/RTO markets on customer costs.
n.
System Lambda
System lambda is the incremental cost of energy of the marginal unit
assuming no system constraints. This metric tracks the trend in marginal fuel
costs and is an important metric since fuel costs represent the largest component of
wholesale energy costs. APPA/ELCON recommends that this information be
based on the single incremental cost of energy per FERC Form 714, rather than as
a component of the locational marginal price as proposed. APPA/ELCON
believes that with a correct definition, system lambda could provide useful
information. Commission Staff agrees with APPA/ELCON that FERC Form 714
provides an appropriate basis for measuring system lambda, and therefore this
information should be included in the report.
o.
Energy Market Price Convergence
i.
ISO/RTO Performance Metrics
Convergence of day-ahead and real-time energy prices provides an
indication of the efficiency of ISO/RTO markets. Since the large majority of
energy settlements and generator commitments occur in the day-ahead market,
day-ahead price convergence with the real-time market ensures efficient dayahead commitments that reflect real-time operating needs. Energy market price
convergence is measured by the absolute value and percentage of the annual
difference between real-time energy market prices and day-ahead market prices.
ii.
Comment
California PUC recommends adding information on virtual bidding, market
liquidity (total participants in physical and virtual trading markets) and quarterly
information on energy traded and ISO/RTO costs allocated to physical trading and
virtual trading in total dollars and per megawatt-hour costs.
iii.
Response
36
Docket No. AD10-5
Commission Staff does not consider trader capabilities and market trends
that support profitable participation to be reflective of ISO/RTO market efficiency.
Likewise, Commission Staff considers the bidding and trading activity of virtual
traders to be based primarily on assessments by traders as to the profitability of
participation as well as their financial capabilities to take advantage of market
shifts. Since the ability of ISOs/RTOs to influence these factors is either nonexistent or extremely limited, Commission Staff does not think it necessary to
gather this information as part of an ISO/RTO performance assessment.
Responding to California PUC’s recommendation for quarterly data, Commission
Staff considers annual information and long-term trends to be the appropriate basis
to evaluate this information. Information for shorter time periods are heavily
influenced by seasonal shifts and operational factors that make an evaluation of
the efficiency of ISO/RTO day-ahead and real-time markets difficult.
Commission Staff does not consider information on ISO costs allocated to
physical and virtual trading to be appropriate, as recommended by California
PUC, since it is not relevant to an evaluation of the efficiency of prices in
ISO/RTO markets.
p.
Congestion Management Metrics
i.
ISO/RTO Performance Metrics
Congestion represents the cost to customers of paying for more expensive
energy because physical transmission line limits do not allow full delivery of leastcost energy. The first congestion management metric, annual congestion costs
divided by the megawatt hours of load served, tracks congestion cost trends
relative to load growth thereby providing an indication of the efficiency of the
overall ISO/RTO system as well as the effectiveness of ISO/RTO efforts to
manage congestion costs through transmission expansion planning and other
efficiency measures. This metric is also influenced by other factors, such as load
trends, and as such its inputs are not entirely within the control of the ISO/RTO.
The second metric, congestion revenues paid divided by congestion charges
expressed as a percentage, tracks the ability of market participants to hedge these
congestion costs, and thereby manage their costs.
ii.
Comments
Duke considers the value of the metric to be limited because of differences
in size/configuration of ISOs/RTOs. NRECA and APPA/ELCON recommend
adding a metric that analyzes the costs of transmission investment and the
associated value of congestion that would be relieved by such investments. The
ISO New England Market Monitor notes that higher congestion costs may make
sense if transmission costs are high and transmission is difficult to build. Various
37
Docket No. AD10-5
commenters 53 recommend a number of additional and more granular metrics
including metrics on day-ahead versus real-time congestion, congestion at
interfaces, hedging shortfalls, changes in Narrow Constrained Areas, congestion
charges at different times and FTR prices. EPSA asserts that the metric needs
context in order to be meaningful and useful. APPA/ELCON endorses measuring
congestion based on the difference between the maximum and minimum
locational marginal price for the hour and calculated as a total, average and
maximum difference for each month and year as well as the change from year to
year. TAPS recommends a metric to assess whether ISOs/RTOs are enabling
load-serving entities to secure long-term transmission rights for new long-term
power supply arrangements. It recommends that the measures reveal the degree to
which load-serving entities are being denied long-term rights for existing
resources for which long-term rights were previously allocated, or to what extent
load-serving entities have been unable to collect the full value of the long-term
rights.
iii.
Response
Commission Staff agrees with commenters that these metrics have limited
usefulness in evaluating data across different ISOs/RTOs since the size and
configuration of each ISO/RTO region differs, sometimes significantly, from each
of the other regions. However, Commission Staff considers the trends in these
metrics over time to be an indication of whether congestion management is
improving in each ISO/RTO. Commission Staff agrees with commenters that the
congestion cost metric will have more explanatory value if it is compared to the
costs of relieving constraints. Therefore, the narrative discussions, with
quantification to the extent possible, should include an analysis of the types of
projects (e.g. transmission, generation, or demand response) and costs of relieving
constraints to facilitate this comparison. The narrative explanations of trends
should also address other relevant factors such as the sub-regional congestion and
FTR prices recommended by commenters. 54
Commission Staff will not adopt the APPA/ELCON locational marginal
price metric since the difference between the maximum and minimum locational
marginal price in an ISO/RTO market reflects a number of factors unrelated to
congestion, such as changes in the fuel mix, system operating conditions and load
53
California DWR, ITC, National Grid, NRECA, OMS and SoCal Edison.
54
As discussed in the transmission planning metric section, the narrative
discussions should include an assessment of the impact of transmission planning
on congestion trends.
38
Docket No. AD10-5
shifts. The difference between maximum and minimum locational marginal price
in an ISO/RTO market therefore will not reliably provide insight into ISO/RTO
performance. Commission Staff believes that the hedging data, with narrative
explanation, will provide TAPS with information on ISO/RTO performance with
respect to providing the full value of the long-term transmission rights of loadserving entities for new and existing capacity.
q.
Resource Availability Metric
i.
ISO/RTO Performance Metric
Resource availability is an indication of ISO/RTO market efficiency and
cost management. Generator availability, measured as the unforced 55 generator
capacity available in the ISO/RTO markets, is a measure of generator
responsiveness when the generator owner has indicated that the generation should
be available. Higher generator availability can result in the commitment of fewer
peak generators (or the importation of peak supplies) that potentially have high
costs, thereby resulting in reduced costs. Demand resource availability measures
the availability of demand response when called on to perform and when tested for
its capability to meet capacity requirements.
ii.
Comments
Some commenters 56 argue that the inputs to this metric are beyond
ISO/RTO control and therefore that generator availability is not suitable as a
performance metric. DC Energy proposes metrics to measure cost savings
associated with improved outage statistics and more granular data on availability
by unit type and years of service. California PUC endorses several metrics
measuring forced outage rates and ratable time available for generators. Ohio
Consumer Counsel and OMS support including generator availability information
by generator type and fuel. EPSA recommends including additional information
on out-of-merit dispatch and uplift payments, arguing that out-of-merit dispatch
indicates the transparency and efficiency of the market and that uplift payments
mask inadequate modeling by the ISO/RTO or inefficient pricing protocols. OMS
and EPSA consider out-of-merit information to be a measure of the efficient daily
operation of the ISO/RTO market. NEPGA supports these positions. Ohio
55
Unforced capacity represents the capacity of a generator adjusted for
planned outages.
56
Allegheny, CAISO, Mid-American, Midwest ISO TOs, NECPUC, and
NEPOOL.
39
Docket No. AD10-5
Consumer Counsel recommends a metric to measure the amount of price
responsive demand, by sector, that is attributed to advanced metering
infrastructure.
iii.
Response
While, as commenters note, generator availability is beyond the control of
ISOs/RTOs, this metric is worth tracking for the following reasons: (1) it is a
major factor impacting consumer costs, and therefore should be included in a
comprehensive assessment of consumer costs; and (2) generator availability as
measured by planned outages can be influenced by ISO/RTO incentives for
greater availability, and therefore it provides an indication of the effectiveness of
ISO/RTO programs. Commission Staff agrees that the cost savings information
proposed by DC Energy provides useful information. This information should be
included in each ISO/RTO narrative explanation since not all ISOs/RTOs have the
data needed for the metric. Commission Staff will not request more granular
information by unit type or age since the purpose of the metric is to measure the
effectiveness of ISO/RTO programs to increase the availability of all generators.
Commission Staff also does not consider metrics on forced outages and ratable
time available of generation units to be appropriate, as recommended by California
PUC and OMS. Such metrics measure generator activities over which ISOs/RTOs
have no influence.
Responding to OMS, EPSA and NEPGA, Commission Staff considers outof-merit dispatches to be complex events caused by a number of factors; as such,
they do not measure ISO/RTO performance unless these factors can be assessed
and taken into consideration. For example, if out-of-merit dispatch is required
when generators cannot change output quickly enough to accommodate changes in
demand over five-minute dispatch intervals, and additional, more expensive units
must be committed, both generator responsiveness and performance as well as
ISO/RTO coordination play a role in managing these events. At the same time,
however, Commission Staff agrees with EPSA and NEPGA that these events,
particularly if they are frequent and result in the commitment of many generators,
can subvert the efficiency benefits of ISO/RTO markets. For this reason, narrative
explanations should be included in each section of each ISO/RTO document that
discuss the trends in out-of-merit dispatch and progress made toward reducing
these events and improving market efficiency.
40
Docket No. AD10-5
Commission Staff agrees with commenters 57 that the performance of
demand response resources needs to be addressed for a comprehensive assessment
of resource availability, and therefore Commission Staff supports the inclusion of
information and data, where available, on demand response performance.
Commission Staff agrees with the Ohio Consumer Counsel that additional
information on the status of advanced meter infrastructure programs and their
impact on price responsive demand availability would be appropriate in the
narrative discussions. However, Commission Staff does not consider the cost
measures endorsed by these commenters to be appropriate since the cost of
projects do not reflect ISO/RTO market performance.
r.
Fuel Diversity
This metric is defined to be the percentage mix of fuel types installed and
available (capacity fuel diversity) and consumed (generation fuel diversity) in the
ISO/RTO region. Fuel diversity provides an indication of the capability in an
ISO/RTO market to integrate fuels with different characteristics, such as lower
costs or lower environmental impacts, and how this capability compares to the fuel
mix of energy consumed in ISOs/RTOs. A higher percentage consumption of
low-cost fuels, compared to their capability, would indicate that Security
Constrained Economic Dispatch is maximizing output of lower cost fuels. SoCal
Edison recommends adding information on the total MMBtus consumed in each
fuel class. Commission Staff considers the percentage break-down of fuels to be a
sufficient indication of fuel diversity and therefore additional information on the
physical amounts of fuel consumed is not necessary. Commission Staff
recognizes that some ISOs/RTOs may be resource-constrained due to factors
beyond their control such as geographic locations and environmental regulations
and therefore comparisons across ISOs/RTOs may reflect factors other than
performance. Commission Staff expects that such constraints would be discussed
in the narrative explanations, thereby providing context for the performance
results.
s.
Renewables Metrics
i.
ISO/RTO Performance Metrics
57
DC Energy, Dominion, EPSA, ISO New England Market Monitor, New
York PSC, Ohio PUC and Steel Producers.
41
Docket No. AD10-5
Renewable resource 58 penetration in ISO/RTO markets is measured as the
renewable MW share of total energy, measured in megawatt-hours, and the
renewable MW share of total capacity, measured in megawatts. The renewables
metrics indicate the diversity of an ISO/RTO resource mix to meet demand and
capacity requirements. Greater diversity of resources can reduce consumer costs
by providing more supply options and greater planning flexibility for ISOs/RTOs.
ii.
Comments
Commenters 59 consider the metric to be beyond ISO/RTO control and
therefore not suitable as a performance metric. NESCOE notes that the metric is
developed within the states’ energy and environmental planning and policy
frameworks. EEI and NEPOOL assert that the metric is not meaningful since
there are differences in resource mix, fuel prices, congestion, renewable
penetration and capacity factor in each ISO/RTO. AWEA recommends a number
of metrics to measure the time and planning process steps required to integrate
wind resources onto ISO/RTO systems. OMS endorses capacity factor metrics.
The ISO New England Market Monitor and EPSA recommend metrics to
measure the operating performance of renewable resources and measurement of
the payments made to these resources. Ohio Consumer Counsel endorses metrics
on the costs of renewable interconnections and events during which renewable
resources were backed down. Ohio Consumer Counsel also recommends a metric
to record the amount of behind-the-meter distributed generation (renewable and
combined heat and power) and the share of distributed generation in ISO/RTO
energy and capacity markets.
iii.
Response
Commission Staff recognizes that the amount of renewable resources in a
market is not under the direct control of ISOs/RTOs, and that they are influenced
by a number of factors, such as state renewable initiatives. Nonetheless, the
metrics may be a useful indicator of the integration of renewable resources into the
ISO/RTO markets. Commission Staff recognizes that regional differences among
the ISOs/RTOs will result in differing penetration results for renewable resources.
58
Renewable resources are defined to be variable energy resources,
including solar, wind, hydro, geothermal and biomass resources.
59
Allegheny, Midwest ISO TOs, and NESCOE.
42
Docket No. AD10-5
Commission Staff agrees with commenters that additional information
should be provided and quantified to the extent possible, to the extent it is
available. For this reason, Commission Staff is adding the renewables percentage
of capacity as a metric to the list of proposed metrics issued on February 3, 2010.
Additional information would also be appropriate in narrative explanations that
address the specific market products being provided by these resources and the
most significant aspects of their participation including the status of
interconnecting major renewable resources, as recommended by AWEA and Ohio
Consumer Counsel, the frequency of events in which renewable resources were
backed down, as recommended by Ohio Consumer Counsel and information on
capacity factors as recommended by OMS. Commission Staff does not consider
operating performance and cost measures for these resources to be appropriate
since the operating performance and the cost of the projects do not measure
ISO/RTO performance.
t.
Administrative Costs Metrics
i.
ISO/RTO Performance Metrics
The administrative cost metrics measure the ability of ISOs/RTOs to keep
costs within budgeted levels (actual versus budgeted administrative charges
metric) and to manage the growth rate of administrative costs commensurate with
the growth rate of system load (administrative charges cents per megawatt-hour of
load served metric). The components of ISO/RTO administrative costs are capital
costs – capital charges, debt service, interest expense and depreciation expense –
and operating and maintenance costs net of miscellaneous income. By managing
administrative costs, ISOs/RTOs can reduce customer costs.
ii.
Comments
The New Jersey Rate Counsel notes that since costs are not directly
comparable across ISOs/RTOs due to differences in size, function and capacity
mix, ISO/RTO administrative cost metrics must be defined on a disaggregated
basis. 60 The Midwest ISO states that total administrative charges are most closely
related to the geographic scope and load size of each ISO/RTO and that this figure
would not be an appropriate mechanism for comparing ISO/RTO performance
when there is such a wide variation in geographic scope and load size of the
jurisdictional ISO/RTO. The Midwest ISO also notes that the metric fails to
account for the different sets of services offered by each ISO/RTO to their
60
Ohio Consumer Counsel also supports a breakdown of major categories
of administrative costs as well as weather-normalized data.
43
Docket No. AD10-5
members. OMS supports comparing actual and budgeted administrative charges
with a break-down by category of costs. California DWR recommends a
comparison of administrative and operating costs over time to determine
streamlining and duplication in transmission operational and administrative costs
between ISO/RTO and transmission owners.
The Federal Trade Commission endorses an internal operating efficiency
metric that holds ISOs/RTOs accountable for making cost-effective use of
resources such as staff and software to run the grid efficiently. DC Energy also
supports a metric that measures whether administrative costs are justified by the
projects undertaken.
TANC recommends that the Commission strengthen its oversight of the
ISO/RTO budgeting process and makes the specific recommendation that
ISOs/RTOs be required to submit their proposed budgets for Commission
approval and explain any deviations, such as is required for NERC and the
Regional Entities. ODEC asserts there is need for increased transparency in the
budget process. OMS expresses concern that the stakeholder process for budget
reviews requires considerable resources and does not allow for meaningful
participation for some important stakeholders.
iii.
Response
Commission Staff recognizes that administrative costs will vary by
ISO/RTO and that comparisons between ISOs/RTOs may not be meaningful, as
commenters point out. Commission Staff expects that the primary value of this
metric will be in the analysis of long-term cost trends and the narrative
explanations of major variances from budget and that this information will be
responsive to the concerns of commenters. The narrative discussions should
include discussions of major categories of costs 61 as part of the variance analysis,
thereby addressing recommendations of commenters. Also, the narrative
discussions should address the cost-effectiveness of ISO/RTO administrative
resource planning, as recommended by the Federal Trade Commission. Further,
the narrative discussion could explain the effect that load reductions (due to
demand resources, energy efficiency, or other factors) have had on reducing
Mwhs, and thus the calculation of the second metric, so that these effects do not
convey the misimpression that administrative costs per Mwh have increased.
61
We clarify for California PUC and Ohio Consumer Counsel that the
administrative charges include capital investment.
44
Docket No. AD10-5
Commission Staff does not recommend a mandatory budgeting process. As
discussed in the opening sections of this working paper, this is not a rulemaking
proceeding and therefore mandatory requirements are not appropriate.
u.
Customer Satisfaction Metric
i.
ISO/RTO Performance Metrics
The percentage of satisfied members metric and the independent
assessment of customer satisfaction provide indications of the extent to which
ISOs/RTOs provide value to their customers. The independent assessments of
customer satisfaction are based on surveys undertaken by independent, third-party
entities. These surveys analyze customer perspectives on a wide range of
ISO/RTO activities.
ii.
Comments
The Federal Trade Commission endorses a metric to measure
responsiveness to grid users and the retail customers they serve. California DWR
recommends an independent assessment of customer satisfaction with ISO/RTO
performance and customer friendliness. NEPUC and NEPOOL recommend that
satisfaction should be measured for separate functions; transmission planning,
setting of installed capacity requirements, administration of market, market rule
changes, market settlement, market development, and responsiveness to
information requests. New Jersey Rate Counsel recommends assessing customer
satisfaction by major functional area (e.g., operations, transmission planning,
generation planning) and by customer group. Ohio Consumer Counsel supports
assessing customer satisfaction by sector and Massachusetts AG recommends that
information be provided on customer complaints broken down by topic area with
the corresponding ISO/RTO response.
Several comments also recommend additional metrics. NY Transmission
Owners recommend a metric to measure ISO/RTOs’ correct implementation of
tariffs and avoidance of market errors. National Grid endorses a metric to measure
the frequency of price flagging and tariff waiver requests by ISOs/RTOs. DC
Energy recommends a metric to measure the frequency with which market reports
are posted in a timely manner. NEPOOL endorses a metric to measure the
responsiveness of ISOs/RTOs to participant or consumer requests for cost impact
analyses and analyses of market rule revisions. ODEC has a similar
recommendation for responsiveness per the requirements of Order No. 719. 62
62
Wholesale Competition in Regions with Organized Energy Markets, Order No.
719, 73 Fed. Reg. 64,100 (Oct. 28, 2008), FERC Stats. & Regs. ¶ 31,281 (2008).
45
Docket No. AD10-5
NEPGA supports a metric to measure buyer and seller stakeholder support for
ISO/RTO market rule changes.
iii.
Response
In response to comments, Commission Staff considers it appropriate to
include information in the report from independent assessments of customer
satisfaction and satisfaction surveys of specific ISO/RTO functions. The
Customer Satisfaction Metric will be supplemented by a narrative discussion that
should address responses by customer sector and by functional area as well as
referencing key customer activities such as responsiveness to customer requests,
the timeliness of market reports, time to resolve invoice disputes, correct
implementation of tariffs, market errors/price flagging and stakeholder support for
major rule changes, as recommended by commenters.
v.
Billing Controls Metric
This metric indicates the accuracy and integrity of the ISO/RTO billing
process, based on audits conducted according to the Statement on Auditing
Standards No. 70 guidelines set by the American Institute of Certified Public
Accountants. The audits describe the controls, the results of testing of the
ISO/RTO controls, whether the controls were designed to achieve the control
objectives, the auditor’s opinions on the audit and whether the controls that were
tested were operating with sufficient effectiveness to provide reasonable assurance
that the control objectives were achieved. NY Transmission Owners and
Allegheny recommend metrics to measure billing errors. Commission Staff
considers it appropriate that the narrative explanation that accompanies this metric
include a discussion of major billing issues, including billing errors, rather than
reporting an error metric. Such an explanation should provide a more meaningful
assessment of the significant billing issues that may be the root cause of billing
errors.
46
Docket No. AD10-5
IV.
APPENDIX A
LIST OF INTERVENORS AND COMMENTERS
Allegheny Power and Allegheny Energy Supply Company, LLC
(Allegheny)
American Public Power Association and Electricity Consumers Resource
Council (APPA/ELCON)
American Wind Energy Association (AWEA)
California Department of Water Resources State Water Project (California
DWR)
California Independent System Operator Corporation (CAISO)
California Public Utilities Commission (California PUC)
City of Redding, City of Santa Clara and M-S-R Public Power Agency
(Cities/M-S-R)
COMPETE Coalition (COMPETE)
Consumer Commenters 63 (Consumer Coalition)
DC Energy, LLC (DC Energy)
Delaware Municipal Electric Corporation (DEMEC)
Dominion Resources Services, Inc. (Dominion)
Duke Energy Corporation (Duke)
Edison Electric Institute (EEI)
Eastern Massachusetts Consumer-Owned Systems (EMCOS)
Electric Power Supply Association (EPSA)
63
AARP, American Forest & Paper Association, American Municipal
Power, Inc., American Public Power Association, Blue Ridge Power Agency,
Citizen Power, Citizens Utility Board, Coalition of Midwest Transmission
Customers, Connecticut Office of Consumer Counsel, Delaware Municipal
Electric Corporation, Inc., Electricity Consumers Resource Council, Illinois
Attorney General, Industrial Energy Consumers of America, Kennebunk Light &
Power, Maryland Office of People’s Counsel, Modesto Irrigation District,
Municipal Electric Utilities Association of New York, National Consumer Law
Center, NEPOOL Industrial Customer Coalition, New England Public Power
Association, New York Association of Public Power, Office of the People’s
Counsel for the District of Columbia, Ohio Partners for Affordable Energy,
Pennsylvania Office of Consumer Advocate, PJM Industrial Customer Coalition,
Public Citizen, Public Power Association of New Jersey, Public Utility Law
Project of New York, Inc., and Virginia Citizens Consumer Council.
47
Docket No. AD10-5
Federal Trade Commission (Federal Trade Commission)
FirstEnergy Service Company (FirstEnergy)
Coalition of Midwest Transmission Customers and PJM Industrial
Customer Coalition (Industrial Customers)
Internal Market Monitor of ISO New England Inc. (ISO New England
Market Monitor)
ISO/RTO Council (IRC)
Independent Power Producers of New York, Inc., (IPPNY)
International Transmission Company (ITC)
Maine Public Utilities Commission (Maine Commission)
Public Service Commission of Maryland (Maryland PSC)
Attorney General of the Commonwealth of Massachusetts (Massachusetts
AG)
MidAmerican Energy Company (MidAmerican)
Midwest Independent Transmission System Operator, Inc. (Midwest ISO)
Midwest ISO Transmission Owners (Midwest ISO TOs)
Missouri Public Service Commission (Missouri Commission)
Modesto Irrigation District (MID)
National Association of Regulatory Utility Commissioners (NARUC)
National Grid USA (National Grid)
National Rural Electric Cooperative Association (NRECA)
New England Conference of Public Utility Commissions (NECPUC)
New England Power Generators Association, Inc. (NEPGA)
New England Power Pool Participants Committee (NEPOOL)
New England States Committee on Electricity (NESCOE)
New Jersey Division of Public Advocate, Division of Rate Counsel (New
Jersey Rate Counsel)
New Hampshire Public Utilities Commission (New Hampshire PUC)
New York State Public Service Commission (New York PSC)
Indicated New York Transmission Owners (NY Transmission Owners)
Public Utilities Commission of Ohio (Ohio PUC)
Office of the Ohio Consumers’ Counsel (Ohio Consumer Counsel)
Old Dominion Electric Cooperative (ODEC)
Organization of MISO States (OMS)
Pacific Gas and Electric Company (PG&E)
Pennsylvania Public Utilities Commission (Pennsylvania PUC)
Pepco Holdings, Inc. (PHI Companies)
PJM Power Providers Group (P3)
Connecticut Municipal Electric Energy Cooperative, Massachusetts
Municipal Wholesale Electric Company and New Hampshire Electric
Cooperative (Public Systems)
48
Docket No. AD10-5
Retail Energy Supply Association (RESA)
Southern California Edison Company (SoCal Edison)
Steel Producers Association (Steel Producers)
Transmission Association of Northern California (TANC)
Transmission Access Policy Study Group (TAPS)
Tyrone Christy, Vice Chairman, Pennsylvania Public Utilities Commission
(Commissioner Christy)
United Illuminating Company (United Illuminating)
Viridity Energy, Inc. (Viridity)
Westar Energy Inc. (Westar)
49
Docket No. AD10-5
V.
APPENDIX B
50
Appendix B
ISO/RTO Performance Metrics
Performance Metric
Reliability
A. National or Regional
Reliability Standards
Compliance
Docket No. AD10-5
Specific Metric(s)
1. References to which Electricity Reliability Organization (ERO) and Regional Reliability
Organization (RRO) standards are applicable to each ISO/RTO
2. Number of violations self-reported
3. Number of violations identified as RRO or ERO audit findings
4. Total number of violations
5. Severity level of each violation
6. Compliance with operating reserve standards
7. Unserved energy (or load shedding) caused by violations
B. Dispatch Reliability
1. Balance Authority Ace Limit (BAAL) OR// CPS1 and CPS2
2. Number of hours of transmission load reliefs (of severity level 3 or higher) or unscheduled
flows
3. Energy Management System (EMS) availability
C. Operational Planning –
Load Forecast Accuracy
Actual load as a percentage variance from forecasted load (separate metrics for peak and
valley metric)
D. Wind Forecasting
Accuracy
E. Unscheduled Flows
Actual wind availability compared to forecasted wind availability
F.
1. Percentage of > 200kV planned outages of 5 days or more that are submitted to ISO/RTO at
least 1 month prior to the outage commencement date
Transmission Outage
Coordination
Difference between net actual interchange (actual measured power flow in real time) and the
net scheduled interchange in megawatt hours (total system and major interfaces) and as a
percentage of total flows
2. Percentage of planned outages studied in the respective ISO/RTO Tariff/Manual
1
Performance Metric
Specific Metric(s)
established timeframes
3. Percentage of > 200 kV outages cancelled by ISO/RTO after having been previously
approval
4. Percentage of > 200kV outages (both planned and unplanned) with less than 2 days notice
G. Long-Term Reliability
Planning – Transmission
1. Number of facilities approved to be constructed for reliability purposes
2. Percentage of approved construction on schedule and completed
3. Performance of Order 890 planning process related to:
a. Completion of reliability studies
b. Completion of economic studies
4. Impact of Demand Response on Infrastructure Investment
H. Long-Term Reliability
Planning – Resources
1. Processing time for generation interconnection requests
2. Actual reserve margins compared with planned reserve margins
3. Demand response megawatts as percentage of total capacity
4. Percentage of generation outages cancelled by ISOs/RTOs
5. Number and capacity of generation reliability must run contracts
6. Demand response megawatts as percentage of total ancillary services
I.
Infrastructure Investment
– Interconnection and
Transmission Process
Metrics
1. Number of requests
2. Number of studies completed
3. Average age of incomplete studies
4. Average time for completed studies
5. Average cost of each type of study completed (e.g., feasibility study, system impact study
and facility study)
2
Performance Metric
J.
Special Protection
Systems
Specific Metric(s)
1. Number of special protection systems
2. Percentage of special protection systems that responded as designed when activated
3. Number of unintended activations
Performance Metric
Market Benefits
A. Market Competitiveness
Specific Metric(s)
1. Price Cost Mark Up
2. Generation Net Revenues
3. Percentage of hours offers are capped due to mitigation
B. Market Pricing
1. Load-Weighted Locational Marginal Prices
2. Components of Total Power Costs based on Load-Weighted Locational Marginal Prices
(e.g. fuel costs, transmission charges, RTO costs, etc.)
3. Load-Weighted, Fuel-Adjusted Locational Marginal Prices
4. Impacts of Demand Response on Market Prices
C. System Lambda
System Lambda (on marginal unit); unconstrained energy portion of system marginal cost
D. Energy Market Price
Convergence
1. Absolute dollar difference between day-ahead and real-time prices
E.
2. Percentage difference between day-ahead and real-time prices
Congestion Management 1. Congestion charges per megawatt hour of load served
2. Percentage of congestion dollars hedged through ISO/RTO-administered congestion
management markets
F.
Resource Availability
1. 1 – RTO forced outage rate
2. Demand Response Availability
G. Fuel Diversity
Fuel diversity in terms of energy, installed capacity and actual production
3
Performance Metric
H. Renewables
Specific Metric(s)
1. Renewable megawatt hours as a percentage of total energy
2. Renewable megawatts as a percentage of total capacity.
Organizational Effectiveness
A. Administrative Costs
1. Annual actual ISO/RTO administrative charges to members compared with budgeted
administrative charges
2. Annual actual ISO/RTO administrative charges to members as cents per megawatt hour of
load served
B. Customer Satisfaction
1. Percentage of satisfied stakeholders
2. Independent assessment of stakeholder satisfaction
C. Billing Controls
SAS 70 audit scope (e.g. Type 1 or Type 2 audit) and results
4
| File Type | application/pdf |
| File Title | Federal Energy Regulatory Commission |
| Author | jshmt5 |
| File Modified | 2010-10-22 |
| File Created | 2010-10-20 |