All comments and responses

Attachment 16_All comments and responses_8 16 13.pdf

Willingness to Pay for Improved Water Quality in the Chesapeake Bay (New)

All comments and responses

OMB: 2010-0043

⚠️ Notice: This form may be outdated. More recent filings and information on OMB 2010-0043 can be found here:

2014-04-02 - No material or nonsubstantive change to a currently approved collection

Document [pdf]

Download: pdf | pdf

Attachment 16
Public Comments and Response

Public Comments on Federal Register Notices and Response
A. First Round of Public Comments to 77 FR 43822
1) Utility Water Act Group
Response
2) Coalition of 18 Interest Groups
Response
3) Food and Water Watch
Response

B. Second Round of Public Comments to 78 FR 9045
4) Coalition of 23 Interest Groups
Response
5) Utility Water Act Group
Response
C. Third Round of Public Comments to 78 FR 38713
6) Coalition of 20 Interest Groups
Response
7) Utility Water Act Group
Response
8) Natural Resource Defense Council
Response

D. References

Attachment 16
Public Comments and Response

A.
First Round of Public Comments to 77 FR 43822
Open May 24, 2012
Closed July 23, 2012
Extended
Open July 26, 2012
Closed Aug 27, 2012
Comments:
1) Utility Water Act Group
2) Coalition of 18 Interest Groups
3) Food and Water Watch

Attachment 16
Public Comments and Response

1-1

1-2

Attachment 16
Public Comments and Response

1-3

Attachment 16
Public Comments and Response

1-4

1-5

Attachment 16
Public Comments and Response

1-6

Attachment 16
Public Comments and Response

1-7

1-8

Attachment 16
Public Comments and Response

1-9

Attachment 16
Public Comments and Response

1-10

Attachment 16
Public Comments and Response

1-11

1-12

Attachment 16
Public Comments and Response

1-13

Attachment 16
Public Comments and Response

1-14

Attachment 16
Public Comments and Response

RESPONSES TO COMMENT SET 1: Utility Water Act Group
1-1

Thank you for the detailed comments.

1-2

EPA extended the comment period by 30 days in order to accommodate review of supporting materials.

1-3

See Section 2a of Part A this ICR for a discussion of the purpose of the study.

1-4

Again, see Section 2a of Part A of this ICR for a detailed discussion of the purpose of the study. ‘

1-5

EPA recognizes that hypothetical bias is a potential concern in stated preference (SP) surveys and takes
this concern seriously. In general, SP methods have “been tested and validated through years of
research and are widely accepted by federal, state, and local government agencies and the U.S. courts
as reliable techniques for estimating nonmarket values” (Bergstrom and Ready 2009, p. 26). A recent
meta-analyses of the stated preference literature also concludes that hypothetical bias may not always
be a significant concern (Murphy, et al. 2005).
To reduce the potential for hypothetical bias in this survey EPA has consulted with experts and drawn
from peer reviewed literature to address it in the survey design. For example, the survey explicitly
incorporates elements that allow mitigation of hypothetical bias, such as the use of reminders about
budget constraints (akin to the cheap talk language in Cummings and Taylor 1999; List 2001). These
features of survey design are shown to minimize hypothetical bias in experimental settings. The text
used in this survey has undergone thorough testing with participants in focus group and one-on-one
interviews. EPA believes that the steps taken during survey development and testing have largely
mitigated the potential for hypothetical bias. See Section 2d of Part B of this ICR for more information
on how we address hypothetical bias.
Similarly, EPA recognizes the potential for households to exhibit yea-saying and to overstate or
understate their true WTP in order to influence decisions informed by survey data. Survey and study
design choices can mitigate yea-saying. The use of mail survey rather than face-to-face interview has
been shown to decrease the social pressure that may influence a respondent to provide a response
deemed desirable (Dillman 2000). This survey also employs a conjoint choice framework, where
respondents must consider the trade-offs between a status quo and two policy options. Respondents
are asked to make a discrete choice among three unranked options rather than a simple yes or no.
These options vary in terms of the levels of five environmental attributes (plus cost). In this choice
experiment framework it is has been shown that the likelihood for yea-saying and strategic responses is
less prominent (Blamey and Bennett 2001, Collins and Vossler 2009).
In addition, in order to identify such respondents EPA includes debriefing questions at the end of the
survey to identify respondents who might believe that protecting the environment is important no
matter the cost. Sensitivity analysis will be used to examine if and how responses to these debriefing
questions influence responses. Again, Section 5(b) of Part B of this ICR provides a detailed response.
EPA also recognizes the potential for non-reponse bias and the impacts it could have on the data
analysis. First, EPA is taking steps to obtain the highest possible response rate, thereby mitigating nonresponse bias. Specifically, EPA is also following the Dillman tailored design method (Dillman 2008) for

Attachment 16
Public Comments and Response

mail surveys which includes an introduction letter preceding the survey, a reminder post card, and
second mailing of the survey, and a reminder letter following the second survey.
EPA will also administer a non-response bias study survey (Attachment 11) in both the pre-test and full
survey in order to examine whether or not respondents are systematically different from nonrespondents (see OMB 2006). In the non-response bias survey, households that do not return the
survey will be randomly sampled to receive a short questionnaire by mail. The questionnaire will elicit
basic demographic information as well as a few short questions regarding awareness and the reasons
they did not complete the survey. Responses to these questions will be used to examine whether
respondents are systematically different from non-respondents. See Section 2(c) of Part B of the ICR for
a description of the non-response bias study.
1-6

It is impossible to know the magnitude of nonuse values prior to conducting this study. While
information is available in Bockstael, McConnell and Strand (1989) on the potential value of water
quality improvements in the Watershed, the study is based on a small sample of Bay-area residents, and
provides limited information on a broader set of benefits attributable to water quality improvements.

1-7

Standard survey development protocols have been used to develop the survey. See Section 3(c) of Part
A for a discussion of background information.

1-8

In response to peer review comments from academic experts in stated preference methods, EPA is now
only modeling willingness to pay for improvements in bay water clarity, striped bass, blue crab, oyster
populations, and the quality of lakes in the watershed. This was previously referred to as the “endpoint”
version of the survey. These attributes were chosen based on extensive focus groups and interviews as
the environmental features that are most salient to the general public. Furthermore, EPA and NOAA
models predict that these features will be impacted by the TMDL. The stated preference survey outlined
in the ICR does not estimate the benefits of the TMDL directly; rather this survey is designed to value
generic status quo and policy options that result in changes in the environmental attributes. As part of
the experimental design, respondents are presented with hypothetical changes in these attributes and
cost. In other words, the hypothetical levels associated with each of the attributes and costs in the
survey vary across respondents (see Section 2(d) of Part B). This allows us to identify the parameters
and estimate a range of values associated with different scenarios. The variation in costs across
programs is not intended to reflect the costs of the TMDL, but rather the likely range of values
respondents hold for the options, as found in extensive focus groups and interviews. The parameters
estimated from respondents’ choices to these hypothetical scenarios will then be used to estimate the
benefits of the TMDL incremental to the baseline.

1-9

The survey does remind respondents to consider other things they may spend their money on, like food,
clothing, etc., so that they fully consider their budget constraint before making choices. However,
respondents are also reminded several times that all other factors (including employment) are held
constant across options. In other words, the survey only assesses the value people hold for the
attributes specified in the choice experiments. EPA believes that focusing on this subset of factors will
lead to a conservative but more reliable estimate of total benefits. EPA proposes to administer three
versions of the survey - an increasing baseline, decreasing baseline and constant baseline - in order to
estimate benefits of environmental improvements relative to a range of baseline scenarios.

1-10

EPA conducted 10 focus groups and 59 one-on-one interviews with individuals within and outside the
Watershed in order to test their level of understanding of the materials included in the survey (OMB
16

Attachment 16
Public Comments and Response

Control Number 2090-0028). We used this standard survey design protocol to identify the most salient
environmental endpoints that will be affected by the TMDL.
1-11

See Sections 2(b) and 5(b) of Part B of the ICR for the survey implementation and econometric analysis
approach to be used in the survey project.

1-12

Again, the EPA disputes the idea that the stated preference method does not have the ability to collect
information with, “quality, objectivity, utility, integrity” on the foundation that these methods are
largely accepted as a valuable tool among those seeking to understand the benefits of changes to
nonmarket goods. The use and nonuse willingness-to-pay estimates generated from this research will
provide a more well-rounded evaluation of future pollution reduction programs in the Chesapeake Bay,
contributing to the quality, objectivity, and integrity of information the EPA will disseminate.

1-13

We appreciate the attention to these details addressed by UWAG and can assure them that any errors
within the experimental design have been rectified.

1-14

EPA believes this study will allow public values and opinions to be included in the decision-making
process for the Chesapeake Bay. Using current econometric methods, this study will provide unique,
policy relevant information about what, if any, further actions are called for in the Chesapeake Bay.

Attachment 16
Public Comments and Response

2-1

2-2

2-3

Attachment 16
Public Comments and Response

2-4

2-5

2-6

Attachment 16
Public Comments and Response

2-7

2-8

2-9

2-10

Attachment 16
Public Comments and Response

2-11

Attachment 16
Public Comments and Response

2-12

2-13

2-14

2-15

Attachment 16
Public Comments and Response

2-16

2-17

2-18

2-19

2-20

2-21

Attachment 16
Public Comments and Response

RESPONSES TO COMMENT SET 2: Coalition of 18 Interest Groups (C18)
2-1

A complementary study of the costs of the TMDL is being conducted by EPA’s Chesapeake Bay Program
Office and will be issued by EPA after a peer-review is complete.

2-2

No response required.

2-3

No response required.

2-4

EPA recognizes that hypothetical bias is a potential concern in stated preference (SP) surveys and takes
this concern seriously. In general, SP methods have “been tested and validated through years of
research and are widely accepted by federal, state, and local government agencies and the U.S. courts
as reliable techniques for estimating nonmarket values” (Bergstrom and Ready 2009, p. 26). A recent
meta-analyses of the stated preference literature also concludes that hypothetical bias may not always
be a significant concern (Murphy, et al. 2005).
To reduce the potential for hypothetical bias in this survey EPA has consulted with experts and drawn
from peer reviewed literature to address it in the survey design. For example, the survey explicitly
incorporates elements that allow mitigation of hypothetical bias, such as the use of reminders about
budget constraints (akin to the cheap talk language in Cummings and Taylor 1999; List 2001). These
features of survey design are shown to minimize hypothetical bias in experimental settings. The text
used in this survey has undergone thorough testing with participants in focus group and one-on-one
interviews. EPA believes that the steps taken during survey development and testing have largely
mitigated the potential for hypothetical bias. See Section 3(b) of Part A of this ICR for more information
on how we address hypothetical bias.
EPA also recognizes the potential for non-reponse bias and the impacts it could have on the data
analysis. First, EPA is taking steps to obtain the highest possible response rate, thereby mitigating nonresponse bias. Specifically, EPA is also following the Dillman tailored design method (Dillman 2008) for
mail surveys which includes an introduction letter preceding the survey, a reminder post card, and
second mailing of the survey, and a reminder letter following the second survey.
EPA will also administer a non-response bias study survey (Attachment 11) in both the pre-test and full
survey in order to examine whether or not respondents are systematically different from nonrespondents (see OMB 2006). In the non-response bias survey, households that do not return the
survey will be randomly sampled to receive a short questionnaire by mail. The questionnaire will elicit
basic demographic information as well as a few short questions regarding awareness and the reasons
they did not complete the survey. Responses to these questions will be used to examine whether
respondents are systematically different from non-respondents. See Section 2(c) of Part B of the ICR for
a description of the non-response bias study.
EPA agrees that it challenging to measure complex environmental commodities. Standard survey design
protocols were followed in developing the survey. As such, EPA conducted 10 focus groups and 72 oneon-one interviews with individuals within and outside the Chesapeake Bay Watershed in order to test
their level of understanding of the materials included in the survey (OMB Control Number 2090-0028).
We used this standard protocol to identify the most salient environmental commodities that will be
affected by the TMDL. Limiting the survey to those policy outcomes (i.e., water clarity, striped bass,
25

Attachment 16
Public Comments and Response

oysters, blue crabs, and lake water quality) is conservative but we can be confident in the benefits we do
capture from the survey.
2-5

EPA believes the survey has practical utility, as required by the Paperwork Reduction Act. The results of
the study will be made available to state and local governments which they may use to better
understand the preferences of households in their jurisdictions and the benefits they can expect as a
result of meeting the TMDL. Finally, stakeholders and the general public will be able to use this
information to understand the social benefits of improving water quality in the Chesapeake Bay
Watershed to accompany the cost information also being developed by EPA. EPA also believes that the
survey meets OMB’s information quality guidelines. We agree that a number based on a poor quality
survey is inferior to no number at all. Therefore, EPA is using standard survey design protocols in the
design and implementation of the survey, including extensive focus group and interview testing, a pretest, and a non-response bias follow-up analysis.

2-6

The attributes on the survey (i.e., water clarity, striped bass, oysters, blue crabs, and watershed lake
conditions) were chosen because water quality and ecological modeling show that they will be affected
by the nutrient and sediment reduction targets in the TMDL. EPA’s National Center for Environmental
Economics has been working closely with water quality modelers in the EPA Chesapeake Bay Program
Office and the Office of Research and Development to quantify the impact of the TMDL on the chosen
attributes.
EPA has also been working closely with ecosystem modelers in NOAA’s Chesapeake Bay Office and
National Marine Fisheries Service’s Office of Habitat Conservation. Specifically, NOAA’s modelers have
provided assistance with the eco-system based fishery models "Ecopath with Ecosim" and "Atlantis."
These consultations have been instrumental in examining the ecological impacts of reducing nutrient
and sediment loads to the Bay of the ecosystem-based fishery models and will allow EPA to more
accurately translate the values people place on the various attributes of the Chesapeake Bay highlighted
in the survey to benefits estimates associated with the TMDLs.

2-7

The survey is indeed framed in a way to elicit “willingness to pay for generic improvements in water
quality.” This allows EPA to estimate the parameters for a range of policy outcomes, which will then be
used to estimate a “benefits curve.” To allow for a range in outcomes, EPA describes conditions in 2025
with the current programs in place and have developed three survey versions with different
hypothetical future baseline conditions (i.e., with no additional programs), where environmental quality
is increasing, decreasing, or constant, as described in Section 5(b) of Part B of this ICR. The benefits
curve will be used to estimate the incremental benefits of the TMDL relative to the most accurate
baseline as predicted by the water quality and ecological models developed by EPA and NOAA.
Sensitivity analyses will be conducted on the results of the survey to examine the effect of uncertainty in
future levels of the environmental conditions, under both the baseline (i.e., without the TMDL) and
TMDL scenarios.
Flexibility in the baseline and policy outcomes are important in this case because the Chesapeake Bay
TMDL allows for adaptive management and additional offsets if the required nutrient reductions are not
being met. So as population in the watershed grows over the future and land use patterns change,
these survey data will still be useful in estimating the benefits of nutrient and sediment reductions in the
Chesapeake Bay.
26

Attachment 16
Public Comments and Response

2-8

The EPA recognizes that there are other programs and activities that will affect water quality in the
Watershed. For this reason we have included an increasing baseline version of the survey to reflect the
fact that absent new programs it is plausible that conditions will improve in the Watershed under these
existing programs.

2-9

Again, the improving baseline version of the survey captures this scenario.

2-10

See 2-9.

2-11

EPA agrees that improvements to lakes that are not in the Watershed should not be included in the
survey. We have made several modifications to the survey instrument to make it clear that only lakes in
the Watershed should be considered. First, we have enhanced the map at the beginning of the survey
to identify major cities within and outside the Watershed and added the Finger Lakes to the map (which
are clearly marked as being outside the watershed). This helps orient respondents who are considering
whether or not they “use” (i.e., engage in recreation activities) the Watershed. Second, we clearly
describe the Watershed as including lakes and state that water bodies outside of the Watershed will not
be affected by the programs. Finally, we include a follow-up question designed to test their level of
understanding that conditions in lakes outside the watershed will not be affected by the programs
described by the survey.

2-12

In addition to providing an enhanced map of the Watershed we identify which sampled households are
in the Watershed and which are not. Respondents will be told in the cover letter of the survey if their
home address is inside or outside the watershed. See Attachments 5 and 6 for examples of the cover
letters.

2-13

The survey scenarios were designed based on the goal of illustrating hypothetical but realistic policy
scenarios that “span the range over which we expect respondents to have preferences, and/or are
practically achievable” (Bateman et al. 2002, p. 259). In the survey these scenarios are framed as generic
policies in order to estimate the range of benefits for water quality improvements. These benefit
estimates will then be used to estimate the incremental benefits of the TMDL relative to the baseline
(see response 2-7).
The survey provides examples of sources of nutrients, including fertilizers, livestock manure, and
household wastewater. The list is not intended to be comprehensive. As stated above, different
versions of the survey have different baseline assumptions, which will be used in the statistical analysis
to reflect the fact that future conditions in the Bay, absent new programs, are uncertain. EPA agrees
that this baseline uncertainty stems, at least partially, from the fact that the TMDL does not impact
other sources of nutrients and sediments, including air disposition from outside the watershed,
sediments, and hurricanes and ocean currents.

2-14

While the sequence of implementation is unknown the experimental design allows EPA to estimate
benefits for a range of outcomes.

2-15

We have added information on page 11 of the survey to inform respondents that programs will be
implemented over time, with full implementation occurring in 2025.
27

Attachment 16
Public Comments and Response

2-16

A separate analysis of the costs of implementing the TMDLs is being developed by EPA’s Chesapeake Bay
Program Office and will be available upon the completion of peer review.

2-17

EPA agrees and a version of the survey with an increasing baseline is now included in the Information
Collection Request.

2-18

EPA agrees and does not intend to add the total monetized benefit results from this study with results
from other studies, such as those that use revealed preference methods. The results from this study can
be used to isolate nonuse values or used alone as a measure of total monetized benefits.

2-19

EPA carefully reviewed the survey instrument and has corrected typos.

2-20

Please see Section 2(b) of Part B of the ICR for the sampling methodology.

2-21

EPA is using state-of-the-science methods to assess the benefits of the TMDL for the Chesapeake Bay.
As such EPA believes that the results will provide useful information to the public and decision makers
on how society values improvements in environmental conditions in the Chesapeake Bay.

Attachment 16
Public Comments and Response

3-1

3-2

3-3

3-4

3-5

Attachment 16
Public Comments and Response

3-6

3-7

3-8

3-9

3-10

Attachment 16
Public Comments and Response

RESPONSES TO COMMENT SET 3: Food and Water Watch
3-1

Thank you very much for the detailed comments. Stated preference surveys (or surveys to measure
WTP) have been used by a variety of federal agencies to assess the benefits of regulations and federal
activities (see, for example, NOAA 2002; USEPA 2008, 2009a; U.S. Bureau of Reclamation 2012). The use
of stated preferences studies (i.e., WTP studies) is consistent with EPA’s peer-reviewed Guidelines for
Preparing Economic Analyses (USEPA 2010) and OMB Guidelines, Circular A-4 (OMB 2003). The use of a
choice experiment design is consistent with standard practice in the peer-reviewed literature for valuing
environmental resources (see Freeman 2003; Bennett and Blamey 2001; Louviere et al. 2000). The
individual choices reflected in each household survey response are aggregated with other household
responses to estimate a total value for the resource. The stated preference survey is not part of a water
quality trading plan, nor will the results of the survey be used to develop a trading plan. The survey is
designed to estimate the welfare impacts of water quality improvements and will have no bearing on
how those improvements are achieved.

3-2

No response required.

3-3

We agree that the Bay is a complex resource and estimating a total value is challenging. EPA conducted
10 focus groups and 72 one-on-one interviews with individuals within and outside the Watershed.
These standard protocols allowed for testing of individual’s understanding of the materials included in
the survey instrument. This approach was used to identify the most salient environmental resources
that will be affected by the TMDL. Limiting the survey to those outcomes (i.e., water clarity, striped
bass, oysters, blue crabs, and water quality of lakes in the watershed) is conservative, but means that
we are more confident in the benefits we do capture from the survey.

3-4

The study that is referenced (i.e., a citation in Diamond and Hausman 1994 to Desvousges 1993) is
almost 20 years old and uses methods that are no longer considered standard (e.g., use of convenience
samples). It is standard to include debriefing questions to capture various biases that may appear in
survey responses, such as “warm glow.” As such we have included questions to capture respondents
who may be responding in such a way.

3-5

The study that is referenced (i.e., Loomis and White 1996) is a meta-analysis based on older studies,
many of which were unpublished or not peer-reviewed. While examples of implausible survey results
exist, including appropriate debriefing questions, use of focus groups, and pre-testing reduces such
occurrences. This project is based on current survey design methods reflecting careful design choices.
In addition, the survey instrument will be pre-tested with a small sample to determine whether or not
responses are plausible and consistent with economic theory.

3-6

Stated preference surveys capture individual preferences for public goods, that is environmental
resources that are shared by all. The choices individuals make in the experimental setting reflect the
trade-offs, or preferences, for that individual between environmental improvements and costs. By
examining and aggregating individual preferences or choices using the analytical methods described in
Section 5 of Part B of this ICR, the researcher (i.e., EPA) is able to discern a value from the sample of
individual choices for the various environmental improvements (also called “attributes”) in the survey.
The survey clearly states that many households are being asked about their preferences and choices,
and therefore does not imply that any one person would be solely responsible for the program choices.
33

Attachment 16
Public Comments and Response

3-7 and 3-8
The stated preference survey is not part of a water quality trading plan, nor will the results of the survey
be used to develop a trading plan. The survey is designed to estimate the welfare impacts of water
quality improvements and will have no bearing on how those improvements are achieved.
3-9

Stated preference surveys are routinely used in federal agencies to estimate the value of non-market
goods (see, for example, U.S. EPA 2008, 2009a; U.S. Bureau of Reclamation 2012). It is not a method to
determine a “price” for a good to be sold, but rather a method to reflect society’s value of the resource.
There are no plans to “sell” the Chesapeake Bay.

3-10

Enforcement remains an important and relevant goal of the EPA.

Attachment 16
Public Comments and Response

B.
Second Round of Public Comments to 78 FR 9045
Open Feb 7, 2013
Closed March 11, 2013
Comments:
4) Coalition of 23 Interest Groups
5) Utility Water Act Group

Attachment 16
Public Comments and Response

4-1a

4-1b

4-2

Attachment 16
Public Comments and Response

4-3a

4-3b

4-3c

4-3d

4-3e

Attachment 16
Public Comments and Response

4-4

4-5

Attachment 16
Public Comments and Response

RESPONSES TO COMMENTS SET 4: Coalition of 23 Interest Groups (C23)
4-1a.

The purpose of the survey is to value water quality improvements of the type that are expected to
result from the Chesapeake Bay TMDL. While the survey does not refer to Executive Order 13508 or the
Chesapeake Bay TMDLs by name, the range of improvements on the survey cover the improvements
predicted by the Chesapeake Bay Watershed models under the TMDLs. Describing the policy behind the
water quality improvements introduces unnecessary “policy jargon” and would require several more
pages of text in the information section of the survey. In order to maximize response rates we are
keeping the burden placed on the respondent as low as possible by limiting the information sections of
the survey to what respondents need to know to answer the choice questions. So, while the survey
does not discuss the Chesapeake Bay TMDLs by name it is well suited to estimate benefits from the
resulting water quality improvements.

4-1b.

The Northeast Lakes model was designed specifically to model changes in the eutrophication of
freshwater lakes as the result of management practices aimed at improving the water quality of coastal
estuaries. Combining data from the National Lakes Assessment and results from the Spatially
Referenced Regressions On Watershed Attributes (SPARROW) nutrient models, the Northeast Lakes
model uses nutrient loads to watershed streams and rivers to forecast eutrophication of lakes in the
watershed. The Northeast Lakes Model places every lake in the Chesapeake Bay watershed into one of
four eutrophication categories. The “low algae growth” lakes on the survey refer to the lower three
categories. The Northeast lakes model is described in more detail in Moore et al. (2011) and Booth et
al. (2011).

4-2.

No response needed.

4-3a.

The discrepancy between the 764 million number quoted in the comment and the 250 million number
used on the survey is due to the inclusion of juvenile crabs in the larger number. EPA chose to use the
adult spawning population for three reasons. (1) It is more stable from year to year than the total
population because of the vulnerability of juvenile crabs to a variety of environmental factors including
temperature. (2) The adult population is considered to be the harvestable stock and will support the
recreational fishery. (3) The adult population, particularly the females, is related to the number of
young crabs that can be produced each year and is an important indicator of the health of the stock.
(Maryland DNR, http://dnr.maryland.gov/fisheries/crab/dredge.asp) Page 3 of the survey been revised
to clarify that the population refers to adult crabs.

4-3b.

Describing the relevant context for the current conditions and providing respondents references to
target levels set by fishery managers is very important. This language was tested in focus groups and
commented on by the external peer reviewers. The survey was revised in response to feedback from
respondents about the levels and targets, what they mean, and how they were determined. External
peer reviewers reinforced the idea that policy benchmarks do not necessarily help respondents better
understand attribute levels. Conditions in the recent past, which respondents can understand and relate
to recent experiences, provide a more objective and grounded reference point for respondents to
decide what choices are best for them and their household. Therefore on page 3 of the survey,
information is provided on conditions in the early 1990s, with current conditions provided in relation to
this marker. In addition, the early 1990s is the time at which data started being collected at regular
intervals on all the choice question attributes.

Attachment 16
Public Comments and Response

4-3c.

The current number of lakes with low algae levels is based on the results of the Northeast Lakes Model
(see response 1b) which uses EPA’s National Lakes Assessment
(http://water.epa.gov/type/lakes/lakessurvey_index.cfm) to characterize eutrophication levels and
algae growth in freshwater lakes.

4-3d.

Focus group testing of the survey showed that documenting the source of the predictions for policy and
baseline scenarios improved the credibility and consequentiality of the choice questions. Focus group
participants wanted to know the source of the information on the survey. Removing the documentation
for these predictions and replacing it with a description of those predictions as “hypothetical” would
undermine the credibility of the survey instrument, the consequentiality of the choice questions and
produce less reliable results. However, to reinforce the point that the estimates are not certain we have
revised the survey to refer to these estimates as “predictions,” a term more commonly used for
modeled outcomes than “forecast.”

4-3e.

We have added Attachment 14 to the docket that describes how attributes in the choice questions were
modeled and includes documentation for all models used to predict attribute levels under baseline and
policy conditions.

4-4

EPA is aware that some management practices specified in the Watershed Implementation Plans will
not reach their full effectiveness for many years after implementation and EPA will be explicit about
those time lags in the benefit analysis. How to address such time lags is an important and oftenencountered challenge in stated preference study design and an active area of research.
It is generally accepted practice in the stated preference literature to provide stylized information on the
timing of the benefits, estimate WTP for a certain outcome, and then perform ex-post discounting and
sensitivity analysis to account for longer time lags and uncertainty in the environmental outcomes (e.g.,
Alberini et al. 2004, Banzhaf et al. 2006, Cameron and DeShazo 2013). In part, this reflects a choice to
reduce outcome uncertainty that will be implicit, but not separately observable, in survey responses.
Uncertainty in outcomes and differences in timing can then be reflected explicitly in the application of
the results.
Such adjustments are, for example, the standard approach to valuing reduced mortality risks at EPA and
elsewhere. Estimates of the value of statistical life (VSL) from the economics literature whether from
stated preference or revealed preference studies typically focus on immediate risk reductions, but for
many policies there is a lag between changes in exposure and changes in risk. Consistent with guidance
from OMB and EPA, these existing VSL estimates are discounted appropriately to account for the
differences in timing between the study and the policy scenarios.
Still, there are reasons to favor describing a longer time frame for the realization of benefits associated
with policy actions in the survey instrument for this case. First, using a shorter time frame requires
strong assumptions regarding respondents’ discount rates and their perception of the transition of the
survey attributes to long term levels. In addition, using a shorter timeframe for environmental
improvements would be changing aspects of the policy that may be welfare relevant and could
therefore affect willingness to pay.
In light of these factors and to ensure the most rigorous analysis possible, EPA will employ a split sample
design. Consistent with TMDL requirements, all surveys will make clear that practices are put in place by
42

Attachment 16
Public Comments and Response

2025, but the year for which improvements are characterized, the “reference year,” will vary. Half of
the sample will receive the original version of the survey in which 2025 is the reference year for the
attribute levels. The other half of the sample will receive a survey that uses 2040 as the reference year.
EPA will discount WTP estimates from the 2025 version of the survey to make them comparable to 2040
estimates and provide a range generated by two valid but different approaches to stated preference
study design.
We will include debriefing questions on all surveys to test for scenario rejection of the type we
encountered in focus groups. If the pretest results show that a disproportionate number of respondents
reacted negatively to either reference year we will reconsider the split sample design for the full survey.

4-5.

The complementary cost analysis that EPA is conducting is taking into account all management practices
that are incremental to the Chesapeake Bay TMDL, including those in the greater watershed.

Attachment 16
Public Comments and Response

5-1

Attachment 16
Public Comments and Response

5-2

Attachment 16
Public Comments and Response

5-3

Attachment 16
Public Comments and Response

5-4

Attachment 16
Public Comments and Response

5-5

5-6

5-7

5-8

Attachment 16
Public Comments and Response

5-9

Attachment 16
Public Comments and Response

5-10

Attachment 16
Public Comments and Response

RESPONSES TO COMMENTS SET 5: Utility Water Act Group (UWAG)
5-1

In accordance with the Paperwork Reduction Act (44 U.S.C. 3501 et seq.), EPA published a notice in the
Federal Register on May 24, 2012, announcing EPA’s intent to submit this application for a new
Information Collection Request (ICR) to the Office of Management and Budget (OMB), and soliciting
comments on aspects of the information collection request (See Attachment 7 for a copy of the Federal
Register notice, 77 FR 31006). Because certain supporting documents were not available in the docket
for public review during the first 30 days of the comment period, EPA re-opened the comment period
for an additional 30 days beginning on July 26 (77 FR 43822; Attachment 7). Also see docket # EPA-HQOA-2012-0033.
The commenter notes that Attachment 13, the response to comments from the first public comment
period, was not posted to the docket when the second public comment period began. However, the
supporting statement includes a lengthy summary of the comments and EPA’s responses to those
comments received. Attachment 13 was made available within 3 days of the submission of the request
to docket customer service.
The Agency is required to provide notice in the FR and solicit comment in part to: evaluate whether the
collection is necessary for the proper performance of the functions of the Agency; evaluate the accuracy
of the Agency’s burden estimate; and enhance the quality, utility, and clarity of the information to be
collected. The PRA does not require access to the additional underlying documents requested and
therefore, the Agency has met its obligations under the PRA.
However, at the request of OMB EPA is posting additional materials to the docket to supplement the
public record. Reports on focus groups and cognitive interviews conducted during the survey
development phase and a report from peer review of earlier drafts of the survey instrument have been
posted to the docket.

5-2

EPA again refers the commenters to section 2(a) and 2(b) in Part A of the ICR for a discussion of the
purpose of the ICR. In particular as stated in this section, states and their congressional representatives
have expressed a desire to know how practices that reduce nutrients and sediment will benefit their
constituents (see, for example, page 55 of US Congress 2011).

5-3

EPA would like to reiterate that the estimates from this stated preference study will be used in
conjunction with a broader benefit-cost analysis that utilizes several of the other non-market valuation
approaches referred to by UWAG, including recreational demand and hedonic property value methods.
However, as stated in the ICR Part A Section 2(a) and in the literature1, only stated preference methods
can capture non-use values.
It is impossible to know the magnitude of nonuse values prior to conducting this study, hence the need
for the stated preference study proposed in this ICR. While information is available in Bockstael,
McConnell and Strand (1989) on the potential value of water quality improvements in the Watershed,

For example, we refer the reader to A. Myrick Freeman’s book referenced by UWAG in their comments: Freeman, A.
Myrick III. 2003. The Measurement of Environmental Resource Values: Theory and Methods. 2 nd ed. Washington, DC:
Resources for the Future.

Attachment 16
Public Comments and Response

the study is based on a small sample of Bay-area residents, and provides limited information on a
broader set of benefits attributable to water quality improvements.
EPA believes that a stated preference study to measure non-use benefits is particularly appropriate in
the context of the Chesapeake Bay. Similar to the Grand Canyon (as referenced in UWAG’s attached
“Comments on EPA’s Notice of Data Availability of 316(b) Stated Preference Survey), the Chesapeake
Bay is an extremely unique resource. For example, it is the largest estuary in North America.2 Although
water clarity and blue crab in general are not necessarily unique resources, water clarity in the
Chesapeake Bay, and blue crab populations in the Chesapeake Bay, are unique.
Finally, UWAG’s claim that “EPA has specifically acknowledged a lack of direct connection between the
ICR’s environmental attributes and actual TMDL benefits,” is misplaced and stems from a fundamental
misunderstanding of the referred to response 1-8, and to conjoint choice methods more generally. EPA
again refers the commenters to the previous response 1-8. To reiterate, the basic purpose of the stated
preference survey is to estimate a range of values associated with different scenarios. Using
respondents’ choices in the stated preference survey, EPA can then use the estimated parameters to
estimate the benefits of the TMDL incremental to the baseline. This conjoint choice experimental design
allows flexibility, compared to a more conventional contingent valuation approach, for example,
because the benefit estimates can be adjusted to fit a range of assumptions about the policy and
baseline scenarios. These scenarios will be well documented in the final Cost-Benefit Analysis report, to
which the stated preference study proposed in this ICR is one of several inputs.
5-4

EPA recognizes the potential for bias in stated preference surveys and has undertaken efforts to
minimize these biases, as documented in our previous response to comments (Please see comment 1-5)
and as described in of Part A Section 3(b) the ICR. We believe that these measures are sufficient to
identify and overcome significant biases. In general, SP methods have “been tested and validated
through years of research and are widely accepted by federal, state, and local government agencies and
the U.S. courts as reliable techniques for estimating nonmarket values” (Bergstrom and Ready 2009, p.
26).

5-5

See 5-1

5-6

As described above in 5-3, and in prior responses to comments (Please see comment 1-8), the stated
preference study described in the ICR does not estimate the benefits of the TMDL directly, but estimates
a range of values associated with different scenarios. The parameters estimated from respondents’
choices to these hypothetical scenarios will then be used to estimate the benefits of the TMDL
incremental to the baseline. The accuracy of the final benefits analysis does depend upon the accuracy
of modeled outcomes. The applicability of the survey for a specific set of modeled changes is
determined by this range of outcomes, but its validity is not. The EPA survey uses a range of plausible
outcomes to estimate WTP.

5-7

See 5-3

5-8

Econometric analysis of data for choice experiments is well-developed and EPA will use established
econometric techniques, as described in the ICR Part B Section 5(b). The statistical methods, including

Chesapeake Bay Program, http://www.chesapeakebay.net/discover/bay101/facts, accessed May 14, 2013.

Attachment 16
Public Comments and Response

econometric methods for data analysis and the application of the results to the TMDL will be subject to
peer review.
5-9

This project is not designated as a highly influential scientific assessment (HISA). EPA has designated this
project as being “influential scientific information” (ISI) and it is included in the Agency’s Science
Inventory.3 Although the survey is not designated HISA, to ensure that the survey was of high quality
EPA did conduct “periodic consultations with experts in the field throughout the survey development
process,” as stated in ICR (2205.01). EPA also obtained peer review of the survey instrument.
Additional peer review of the statistical methods, including econometric methods will be conducted. The
peer review plan is included in EPA’s Science Inventory database.

5-10

As stated in the prior response to this comment (Please see comment 1-12), EPA disputes the idea that
the stated preference method does not have the ability to collect information with, “quality, objectivity,
utility, integrity” on the foundation that these methods are largely accepted as a valuable tool among
those seeking to understand the benefits of changes to nonmarket goods. The use and nonuse
willingness-to-pay estimates generated from this research will provide a more well-rounded evaluation
of future pollution reduction programs in the Chesapeake Bay, contributing to the quality, objectivity,
and integrity of information the EPA will disseminate.

http://cfpub.epa.gov/si/si_public_record_report.cfm?dirEntryId=239164 accessed on 5/15/13.

Attachment 16
Public Comments and Response

C.
Third Round of Public Comments to 78 FR 38713
Open June 27, 2013
Closed July 29, 2013
Comments:
6) Coalition of 20 Interest Groups
7) Utility Water Act Group
8) Natural Resources Defense Council

Attachment 16
Public Comments and Response

6-1

6-2

6-3

6-4

Attachment 16
Public Comments and Response

6-5

6-6

Attachment 16
Public Comments and Response

6-7

6-8

6-9

Attachment 16
Public Comments and Response

6-10

6-11
6-12
6-13

Attachment 16
Public Comments and Response

RESPONSES TO COMMENTS SET 6: Coalition of 20 Interest Groups (C20)
6-1

Thank you again for raising that point. We are pleased you are satisfied with these revisions.

6-2

The purpose of the survey is to collect data that will enable EPA to value a variety of water quality
improvements relative to a range of baseline scenarios. The claim that the “benefits predicted in the
surveys are not supported by the record” is not applicable in the context of this survey. The survey
instrument itself and predicted levels of environmental attributes therein are never ascribed to the
TMDLs. In order to estimate economic benefits of the Chesapeake Bay TMDLs the experimental design
of the survey must include attribute levels for baseline predictions and policy scenarios that cover the
range relevant to the TMDLs, but need not be limited to that range. EPA’s choice to value changes in
environmental outcomes and a range of attribute levels relative to multiple baselines provides the
flexibility to estimate benefits as expectations of water quality in the Chesapeake Bay evolve. This
Information Collection Request (ICR) and request for public comment pertain to the survey instrument
and stated preference methodology described therein. The data collected from this survey will be
combined with information from numerous other scientific models and studies to estimate the benefits
of the TMDLs. EPA will submit a report of the results for public comment and peer review which will
include predictions of conditions under baseline and policy scenarios with descriptions of our modeling
approach.

6-3

We thank the commenters for pointing out this omission in the footnote on page 3 of the survey. The
referenced Chesapeake Bay Fisheries Ecosystem Model uses output from the Chesapeake Bay
Watershed Models to project a range of attribute levels for striped bass, blue crabs, and oysters. The
footnote on the survey has been revised accordingly. In addition, Attachment 17 has been revised to
clarify how the various models inform the range of attribute levels that will appear in the choice
experiment questions.

6-4

The referenced presentation is noted as “in progress” and the results as “preliminary.” As stated above,
the purpose of the survey is to collect data that will enable EPA to value a variety of water quality
improvements relative to a range of baseline scenarios. In order to estimate economic benefits of the
Chesapeake Bay TMDLs the experimental design of the survey must include attribute levels for baseline
predictions and policy scenarios that cover the range relevant to the TMDLs, but need not be limited to
that range. EPA’s choice to value changes in environmental outcomes and a range of attribute levels
relative to multiple baselines provides the flexibility to estimate benefits as expectations of water
quality in the Chesapeake Bay evolve. This ICR and request for public comment pertain to the survey
instrument and stated preference methodology described therein.
After the survey is implemented, the collected data will be combined with information from numerous
other scientific models and studies to estimate the incremental benefits of the TMDLs relative to a range
of alternative baseline assumptions; this may include scenarios in which only modest changes are
ascribed to the TMDLs. EPA will submit a separate report of the stated preference study results for
public comment and peer review which will include predictions of conditions under baseline and policy
scenarios with descriptions of our modeling approach.

Attachment 16
Public Comments and Response

6-5

Dr. Houde was only one of four experts from the aforementioned consultation (see Attachment 18).
The other three experts stated that striped bass, blue crab, and oyster populations will respond to
nutrient and sediment reductions under the TMDLs in the range of less than 5 to more than 15 years.
Additionally, Dr. Houde’s statement is in regard to the TMDLs. EPA emphasizes that the future
outcomes in the survey are never ascribed to the TMDLs. Since the survey itself is not specifically in the
context of the TMDLs, it does not make sense to tell respondents that the TMDLs may have no effect on
striped bass and blue crab populations.
Finally, this stated preference study includes multiple survey versions with a range of attribute levels for
different baseline conditions and policy outcomes in order to provide flexibility in estimating the
benefits of the TMDLs as our knowledge and understanding of changes in the Chesapeake Bay evolves.
As stated earlier, the purpose of the survey is to collect data that will enable EPA to value a variety of
water quality improvements relative to a range of baseline scenarios. In order to estimate economic
benefits of the Chesapeake Bay TMDLs the experimental design of the survey must include attribute
levels for baseline predictions and policy scenarios that cover the range relevant to the TMDLs, but need
not be limited to that range. EPA’s choice to value changes in environmental outcomes and a range of
attribute levels relative to multiple baselines provides the flexibility to estimate benefits as expectations
of water quality in the Chesapeake Bay evolve. This Information Collection Request (ICR) and request
for public comment pertain to the survey instrument and stated preference methodology described
therein. The data collected from this survey will be combined with information from numerous other
scientific models and studies to estimate the benefits of the TMDL. EPA will submit a separate report of
the stated preference study results for public comment and peer review which will include predictions
of conditions under baseline and policy scenarios with descriptions of our modeling approach.

6-6

The “Northeast Lakes Model” is a particular application of the Northeast United States Spatially
Referenced Regressions on Watershed attributes (NE US SPARROW) model in which predictions of total
phosphorus from the SPARROW model are converted to trophic states for lakes in the Northeast. The
SPARROW model is a well established hydrological nutrient delivery model that has been used by
government agencies and academic researchers since 1997 to analyze the source and effect of nutrient
loading to water bodies and is available for public review. The specifics of the NE US SPARROW model
and its application to freshwater lakes are described in Moore et al (2011). Converting total
phosphorous to trophic states and algae levels is a common practice (e.g. Carlson and Simpson, 1996;
Schindler and Vallentyne, 2008). The particular conversion used for the stated preference survey is
described in USEPA (2009b) and is also available for public review. We have modified Attachment 17 to
clarify the information used to generate a range attribute levels for lakes in the survey.

6-7

Moore, et al. (2011) states, three of the lakes “have virtually reached their nutrient storage capacity”
and as a result less than 1% of the phosphorous that enters those lakes is predicted to remain in them.
That is not to say that reducing phosphorous loadings to those lakes will not have an impact; only that
further increases in loadings are unlikely to increase concentrations above current levels. The paper
goes on to say that the rest of the lakes in the analysis have lower predicted phosphorous
concentrations and will thus retain more of the loadings.

6-8

The conditions shown on the survey are watershed-wide percentages of lakes in the highest
eutrophication category. The watershed covers 64,000 square miles and is large enough by far to be
considered a regional application of the model.
66

Attachment 16
Public Comments and Response

6-9

The Booth et al (2011) paper is included as an additional reference for the NE US SPARROW model.

6-10

The purpose of the survey is to collect data, and not to disseminate information. The survey instrument
and specified changes in the environmental attributes are never claimed to be a result of the TMDLs.
The collected data will enable EPA to value a variety of water quality improvements in the Chesapeake
Bay and Chesapeake Bay Watershed relative to a range of baseline scenarios. After the survey is
implemented and data on household preferences is collected, these data will be combined with
information from numerous other scientific models and studies to estimate the benefits of the TMDLs.
This ICR and request for public comment pertain specifically to the survey instrument and stated
preference study design described therein. EPA will submit a separate report of the stated preference
study results for public comment and peer review which will include predictions of conditions under
baseline and policy scenarios with descriptions of our modeling approach.

6-11

The objective of the focus groups and cognitive interviews was to identify areas of confusion in the
survey instrument in order to develop the clearest and simplest survey. EPA points to the last sets of
cognitive interviews where a near complete survey was tested. In these interviews respondents largely
understood the survey text and questions.

6-12

As documented in the focus group and cognitive interview report, many of the fundamental concerns
with stated preference studies have been thoroughly explored and addressed (e.g., hypothetical bias,
consequentiality, protest responses). Additionally, the survey pre-test will help EPA further ensure that
such issues have been resolved (see Part B, Section 3 of this ICR).
In general, stated preference surveys have produced reliable results for many types of non-market
scenarios in the past under the judgment of federal, state and local government as well as U.S. courts.
For a greater discussion on the recognition of SP methods in policy and economics, EPA again refers the
commenters to Part A Section 3(b).

6-13

While stated preference surveys may be particularly valuable when a resource is unique and impacts are
substantial or irreversible; their utility is not limited to these conditions. In any case the Chesapeake Bay
truly is a unique and iconic resource. Executive Order 13508 describes this estuarine ecosystem as a
national treasure and notes that it is the largest and one of the most biologically productive estuaries in
the nation. EPA refers the commenters to our previous response to comments (Attachment 16, pg 56):
EPA believes that a stated preference study to measure non-use benefits is particularly
appropriate in the context of the Chesapeake Bay. Similar to the Grand Canyon (as referenced
in UWAG’s attached “Comments on EPA’s Notice of Data Availability of 316(b) Stated Preference
Survey), the Chesapeake Bay is an extremely unique resource. For example, it is the largest
estuary in North America.4 Although water clarity and blue crab in general are not necessarily
unique resources, water clarity in the Chesapeake Bay, and blue crab populations in the
Chesapeake Bay, are unique.

Chesapeake Bay Program, http://www.chesapeakebay.net/discover/bay101/facts, accessed May 14, 2013.

Attachment 16
Public Comments and Response

7-1

Attachment 16
Public Comments and Response

7-2

Attachment 16
Public Comments and Response

RESPONSES TO COMMENT SET 7: Utility Water Act Group (UWAG)
7-1

The commenter questions the sufficiency of materials to inform meaningful public review, yet every
example of insufficient material deals with aspects of the benefit analysis outside the stated preference
survey. Most recently, the commenter argues that more information is needed about modeling water
quality and fish population changes that will result from the TMDLs. This information is needed to apply
the results of the stated preference survey to the TMDLs, but is not necessary to conduct the survey.
The purpose of the survey is to collect data that will enable EPA to value a variety of water quality
improvements relative to a range of baseline scenarios. In order to estimate economic benefits of the
Chesapeake Bay TMDLs the experimental design of the survey must include attribute levels for baseline
predictions and policy scenarios that cover the range relevant to the TMDLs, but need not be limited to
that range. EPA’s choice to value changes in environmental outcomes using a range of attribute levels
relative to multiple baselines provides the flexibility to estimate benefits as expectations of water
quality in the Chesapeake Bay evolve. This Information Collection Request (ICR) and request for public
comment pertain to the survey instrument and stated preference methodology described therein. The
data collected from this survey will be combined with information from numerous other scientific
models and studies to estimate the benefits of the TMDLs. EPA will submit a separate report of the
stated preference results for public comment and peer review which will include predictions of
conditions under baseline and policy scenarios.
Nonetheless, the models referenced in Attachment 17 represent the state of the science and in most
cases are, to the best of our knowledge, the only models available to predict the levels of the attributes
that focus group and cognitive interview participants deemed most important. Attachment 17 has been
revised to provide more detail on the models used to develop the ranges for the choice attributes used
on the survey.

7-2

The survey will be administered in two phases: a pretest and the main survey. The purpose of the
pretest is to evaluate the survey instrument and experimental design and make necessary adjustments
to improve the utility, efficiency, and cost effectiveness of the study design. While this is standard
practice in survey administration, we do not expect “drastic changes” to the experimental design
following the pretest.

Attachment 16
Public Comments and Response

July 29, 2013
National Center for Environmental Economics
Office of Policy (1809T)
Environmental Protection Agency
1200 Pennsylvania Ave. NW
Washington, DC 20460
Re: Willingness to Pay Survey for Chesapeake Bay Total Maximum Daily Load:
Instrument, Pre-test, and Implementation”. Docket ID Number EPA-HQ-OA-20120033
The environmental non-profit advocacy organization Natural Resources Defense
Council (NRDC) submits the following comment to the U.S. Environmental
Protection Agency on undertaking a stated preference project to improve estimates
of benefits associated with Chesapeake Bay water quality changes under the TMDL.
The use of Choice Experiments, where individuals are asked to choose their
preferred alternative from a choice set made up of different attributes, including
cost is a versatile and a powerful methodology for estimating the monetary value of
environmental changes (Hanley et al. 2002).

8-1

8-2

There are several valuation studies that have demonstrated the need for including
these non-use values in order to capture the total economic value of environmental
goods. For Example, Carson (1995) noted some 2000 papers or studies where
empirical estimates of non-use values have been made. Concentrating only on onsite users of resources, Chesapeake Bay waters in this case, the benefit estimates
will largely reflect only the values of the users and consist of mostly direct use
values. Consequently, potentially large values held by the remainder of the
population, i.e., the non-users and their values are not captured. It is well known
that non-use values can only be measured using stated preference methods.
Hence, the research is an important step toward providing a comprehensive
estimate of the benefits of water quality improvement in the study region.
Limburg et al. (2002) notes that the complex nature of the water ecosystem often
invokes denying any valuation of the ecosystem services through subjective
preferences. However, the use of endpoint survey, i.e., using ecological endpoints
that are characterized as concrete, tangible, and measurable (Boyd 2007) is an
important step toward using science to evaluate policy. Moreover, the choice
experiment approach also helps address the issue of marginality, where small
changes in the attributes lead to changes in economic value. Such knowledge of the
marginal values of the attributes is useful to link ecosystem research with policymaking (Fisher et al. 2008).
NRDC commends the EPA for its use of economic tools to value environmental
goods and encourages applying such principles to policy design.
71

Attachment 16
Public Comments and Response

Sincerely,
Naveen Adusumilli
Science Fellow, Economist
Natural Resources Defense Council
References
Boyd, J., 2007. The endpoint problem. Resources 165, 26–28.
Carson, R.T. (1995). Contingent Valuation: A Comprehensive Bibliography and
History, Edward Elgar, Northampton, MA.
Fisher, B., et al., 2008. Ecosystem services and economic theory: integration for
policy relevant research. Ecological Applications 18, 2050–2067.
Hanley, N., Mourato, S., Wright, R.E., 2001. Choice modeling approaches: a
superior alternative for environmental valuation? Journal of Economic Survey 15,
435-462.
Limburg, K.E., O'Neill, R.V., Costanza, R., Farber, S., 2002. Complex systems and
valuation. Ecological Economics 41, 409–420

Attachment 16
Public Comments and Response

RESPONSES TO COMMENT SET 8: Natural Resources Defense Council
8-1

EPA agrees that nonuse values should be included in the quantification of benefits of environmental
regulation and we appreciate your comment.

8-2

EPA also agrees that the complexity of ecosystem responses to changing water conditions presents a
challenge to economists quantifying the benefits of improved water quality.
Using environmental endpoints as the attributes in the choice experiment questions limits the number
of environmental outcomes the respondents will consider when choosing their willingness to pay. As a
result this approach is more likely to generate a conservative estimate of total economic value
compared with a survey that uses environmental inputs as attributes and allows the respondents to
form their own expectations for a larger number of environmental outcomes. Using endpoints as
attributes is also more likely to generate more reliable responses, however, because respondents are
considering concrete, tangible, and measurable outcomes that are directly connected to their well-being
when answering the choice questions.

Attachment 16
Public Comments and Response

D. References
Alberini, A., Cropper, M., Krupnick, A. & Simon, N. (2004). “Does the value of a statistical life vary with age and
health status? Evidence from the US and Canada,” Journal of Environmental Economics and
Management 48: 769-792.
Banzhaf, H., Burtraw, D., Evans, D., & Krupnick, A. (2006). "Valuation of natural resource improvements in the
Adirondacks," Land Economics 82(3), 445-464.
Bateman, I., Carson, R., Day, B., Hanemann, M., Hanley, N., Hett, T., Jones-Lee, M., Loomes, G., Mourato, S.,
Ozdemiroglu, E., Pierce, D., Sugden, R., & Swanson, J. (2002). Economic Valuation with Stated
Preference Surveys: A Manual. Northampton, MA: Edward Elgar.
Bergstrom, J.C. & Ready, R.C. (2009). What Have We Learned from Over 20 Years of Farmland Amenity Valuation
Research in North America? Review of Agricultural Economics 31(1), 21–49.
Blamey, R., & Bennett, J. (2001). Yea-saying and validation of a choice model of green product choice. In J.
Bennett & R. Blamey (Eds.), The Choice Modelling Approach to Economic Valuation. Northampton, MA:
New Horizons in Environmental Economics. pp. 178-181.
Bockstael, N.E., McConnell, K.E., & Strand, L.E. (1989). Measuring the Benefits of Improvements in Water
Quality: The Chesapeake Bay. Marine Resource Economics 6, 1-18.
Booth, N.L., Everman, E.J., Kuo, I.L., Murphy, L., & Sprague, L. (2011). A Web-Based Decision Support System for
Assessing Regional Water-Quality Conditions and Management Actions. Journal of the American Water
Resources Association 47(5), 1136-1150.
Cameron, T. A., & DeShazo, J.R. (2013). Demand for Health Risk Reductions. Journal of Environmental Economics
and Management 65(1), 87-109.
Carlson, R.E. & Simpson, J. (1996). A Coordinator’s Guide to Volunteer Lake Monitoring Methods. North
American Lake Management Society.
Cummings, R. & Taylor, L. (1999). Unbiased Value Estimates for Environmental Goods: A Cheap Talk Design for
the Contingent Valuation Method. American Economic Review 89(3), 649-665.
Desvousges, W.H., Johnson, F.R., Dunford, R.W., Boyle, K.J., Hudson, S.P., & Wilson, K.N. (1993). “Measuring
Natural Resource Damages With Contingent Valuation: Tests of Validity and Reliability.” In Contingent
Valuation, A Critical Assessment, J.A. Hausman, ed., pp. 91–164. Amsterdam: Elsevier.
Diamond, P. A. & Hausman, J.A. (1993). Contingent Valuation: Is Some Number Better than No Number?
Journal of Economic Perspectives 8(4), 45-64.
Freeman, A. M. (2003). The Measurement of Environmental and Resource Values: Theory and Methods.
Washington, DC: RFF Press.

Attachment 16
Public Comments and Response

Loomis, J. B. & White, D.S. (1996). Economic Benefits of Rare and Endangered Species: Summary and MetaAnalysis. Ecological Economics 18, 197-206.
Louviere, J., Street, D., Burgess, L., Wasi, N., Islam, T., & Marley, A. (2000). Modeling the choices of individual
decision-makers by combining efficient choice experiment designs with extra preference information.
Journal of Choice Modeling 1(1), 128-163.
Moore, R., Johnston, C., Smith, R., & Milstead, B. (2011). Source and delivery of nutrients to receiving waters in
the northeastern and mid-Atlantic regions of the United States. Journal of the American Water
Resources Association 47(5), 965-990.
NOAA. (2002). Stated Preference Methods for Environmental Management: Recreational Summer Flounder
Angling in the Northeastern United States. Retrieved from
https://www.st.nmfs.noaa.gov/st5/RecEcon/Publications/NE_2000_Final_Report.pdf.
OMB. (2003). Circular A-4, Regulatory Analysis, September 17, 2003. Retrieved from
http://www.whitehouse.gov/omb/circulars_a004_a-4/.
Schindler, D.W. & Vallentyne, J.R. (2008). The algal bowl: Overfertilization of the world’s freshwaters and
estuaries. Edmonton: University of Alberta Press.
U.S. Bureau of Reclamation. (2012). Klamath river basin restoration nonuse value survey. Final Report.
Prepared by RTI International. RTI Project Number 0212485.001.010.
U.S. Congress. House. (2011) Conservation, Energy, and Forestry Subcommittee of the Committee on
Agriculture. Hearing to review the Chesapeake Bay TMDL, agricultural conservation practices,
and their implications on national watersheds. 112th Cong., 1st sess. Washington: GPO, 2011.
Print.
U.S. EPA. (2008). Final Ozone NAAQS Regulatory Impact Analysis. EPA EPA-452/R-08-003.
U.S. EPA. (2009a). Environmental Impact and Benefits Assessment for the Final Effluent Guidelines and
Standards for the Construction and Development Category. EPA-821-R-09-012.
U.S. EPA. (2009b). National Lakes Assessment: A Collaborative Survey of the Nation’s Lakes. U.S. Environmental
Protection Agency. Washington, D.C. Retrieved from
http://water.epa.gov/type/lakes/upload/nla_newlowres_fullrpt.pdf
U.S. EPA. (2010). Guidelines for Preparing Economic Analysis. EPA 240-R-10-001.

File Type	application/pdf
Author	Kelly
File Modified	2013-09-17
File Created	2013-08-16